{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0f48bcda-2645-4cb8-97ec-664c2c01340e",
   "metadata": {},
   "source": [
    "<H1> Introduction </H1>\n",
    "\n",
    "[ProtoRL](https://www.github.com/philtabor/protorl/) is intended to be a lightweight framework (constructed on top of PyTorch) to enable rapid deployment of reinforcement learning agents. It's intended for use by hobbyists on high end desktop hardware - i.e. there is no optimization for deploying on cloud infrastructure.\n",
    "\n",
    "Agents use an actor-learner architecture to leverage highly multithreaded workstations. Actors collect data and feed it to a centralized replay buffer, which is sampled by the learner to update the neural networks. Network parameters are periodically (every time step, by default) downloaded to the actors. Actors have policies that map environment observations to an action; there are a broad variety to choose from.\n",
    "\n",
    "Out of the box, ProtoRL supports Deep Q Learning (dueling and double learning), DDPG, TD3, SAC, and PPO, with more to come in the near future."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "775d1eaa-b2a4-4ffd-9385-cceb17c88630",
   "metadata": {},
   "source": [
    "<H2> Environments </H2>\n",
    "The base installation supports the traditional reinforcement learning environments, as well as the Atari library.\n",
    "\n",
    "Environments are instantiated with the make_env function, that takes a variety of optional inputs.\n",
    " * use_atari: (bool) required to apply wrappers for downsampling, gray scaling, & frame stacking. False by default.\n",
    " * repeat: (int) number of frames to repeat an action. 4 by default.\n",
    " * no_ops: (int) number of operations to do nothing. 0 by default.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "dbd77d00-38e3-4eb5-b17b-e10489a53538",
   "metadata": {},
   "outputs": [],
   "source": [
    "from protorl.wrappers.common import make_env\n",
    "\n",
    "env_name = 'CartPole-v1'\n",
    "env = make_env(env_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e17ccc4d-af02-402c-a0c9-8aed73c44a18",
   "metadata": {},
   "source": [
    "<H2> Networks </H2>\n",
    "The network architectures are generally divided up into bases and heads. The base layer interfaces with the environment, and there are a number of options depending on the type of environment.\n",
    "\n",
    "* CriticBase: concatenates states and actions before feed forward. Relu activations on 2 hidden layers.\n",
    "* LinearBase: Takes observation as input and passes through 2 hidden layers. Relu activations.\n",
    "* AtariBase: 3 layer conv network with relu activations.\n",
    "* LinearTanhBase: Takes observation as input and passes through 2 hidden layers. Tanh activations.\n",
    "\n",
    "Network heads take the inputs from the base layers and output whatever is required.\n",
    "* QHead: outputs the Q values for each action. 1 hidden layer with a relu activation and unactivated output layer.\n",
    "* DuelingHead: outputs both the value and advantage streams for dueling Q learning.\n",
    "* DeterministicHead: outputs a deterministic continuous action with a tanh activation (for ddpg, td3).\n",
    "* MeanAndSigmaHead: outputs a mu and sigma for use in a continuous gaussian distribution.\n",
    "* ValueHead: outputs a single value, intended to represent the value of a state.\n",
    "* SoftmaxHead: outputs a softmax probability distribution.\n",
    "* BetaHead: Outputs an alpha and a beta for use in a beta probability distribution (for PPO)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "592dfde3-26ef-44a1-b576-9e978b24e221",
   "metadata": {},
   "outputs": [],
   "source": [
    "from protorl.networks.base import LinearBase\n",
    "from protorl.networks.head import QHead\n",
    "from torch.nn import Sequential\n",
    "\n",
    "input_dims = env.observation_space.shape\n",
    "hidden_layers = [512]\n",
    "\n",
    "# network will have dims [(*input_dims, 256), (256, 512), (512, n_actions)]\n",
    "base = LinearBase(input_dims=input_dims)\n",
    "head = QHead(n_actions=env.action_space.n,\n",
    "             hidden_layers=hidden_layers)\n",
    "q_online = Sequential(base, head)\n",
    "\n",
    "t_base = LinearBase(input_dims=input_dims)\n",
    "t_head = QHead(n_actions=env.action_space.n,\n",
    "               hidden_layers=hidden_layers)\n",
    "q_target = Sequential(t_base, t_head)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9746cfc-9269-4b3b-a760-f521f104e34c",
   "metadata": {},
   "source": [
    "<H2> Policies </H2>\n",
    "Policies are distinct from actors and agents. This lets actors that live on different threads have different hyperparameters (i.e. each dqn actor could have separate epsilon value.\n",
    "\n",
    "* BetaPolicy: uses a Beta distribution for sampling probabilistic actions. Can return entropy as well as actions and log probs. If you pass in an old_action, it lets you calculate the log probs. For use in PPO.\n",
    "* DiscretePolicy: uses a categorical distribution to sample discrete actions. Can also calculate entropy. For use in PPO.\n",
    "* EpsilonGreedyPolicy: selects a random action some of the time, else selects the max action. For use in DQN.\n",
    "* GaussianPolicy: samples a normal distribution for a continuous action. Can also calculate entropy. For use in PPO.\n",
    "* NoisyDeterministicPolicy: calculates a random noise to add to the output of a deterministic network. For TD3/DDPG."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4891fcb8-5204-422a-bb69-8c65d723b5c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from protorl.policies.epsilon_greedy import EpsilonGreedyPolicy\n",
    "policy = EpsilonGreedyPolicy(n_actions=env.action_space.n, eps_dec=1e-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e4c6dd3-3ca7-4cf9-b4c4-bd985c00e5ea",
   "metadata": {},
   "source": [
    "<H2> Agents </H2>\n",
    "Each agent is comprised of one or more actors and a learner. The actors interact with an environment and generate samples to feed to the centralized replay buffer. The learner samples this data to perform gradient descent on the neural networks. Prioritization is included in ProtoRL and is toggled with a boolean flag."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cabc3770-7b88-4ef1-826d-fb543f450d26",
   "metadata": {},
   "outputs": [],
   "source": [
    "from protorl.agents.dqn import DQNAgent as Agent\n",
    "from protorl.actor.dqn import DQNActor as Actor\n",
    "from protorl.learner.dqn import DQNLearner as Learner\n",
    "\n",
    "dqn_actor = Actor(q_online, q_target, policy)\n",
    "dqn_learner = Learner(q_online, q_target, prioritized=False, lr=1e-4)\n",
    "agent = Agent(dqn_actor, dqn_learner)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "187b8b89-1915-4df2-bbf4-7ff6576b6d79",
   "metadata": {},
   "source": [
    "<H2> Memory </H2>\n",
    "\n",
    "Memory can be a bit tricky. In general, we will always want to save the states encountered, rewards, actions taken, and terminal flags. But that may not be enough. For PPO, for instance, we're going to need to store the log probs of the actions taken, according to the policy at the time. In the code base, I call each of these quantities of interest a field.\n",
    "\n",
    "Memory is handled by a single class that can take any number of fields. It's completely generic in that sense, but it does have strong defaults. It defaults to state, action, reward, new state, terminal flag, with appropriate data types for each of these fields.\n",
    "\n",
    "The memory object is instantiated through the use of an initialization function. The initialize memory function handles the basic type of memory we'll most often want, where we only have to supply the required information. Required arguments are: \n",
    "* obs_shape: list\n",
    "* n_actions: int\n",
    "* max_size: int\n",
    "* batch_size: int.\n",
    "\n",
    "The action space variable (optional, defaults to 'discrete') is there to account for the fact that we have a single memory class that can work for discrete and continuous action spaces. \n",
    "\n",
    "Prioritization is handled through the sum-tree data structure (e.g. proportional sampling) and is controlled through two hyperparameters. Alpha denotes the degree of prioritization; 0 is uniform 1 is fully prioritized. Beta controls the degree of importance sampling for the weights of the neural network. 0 is no weighting and 1 is fully weighted. \n",
    "\n",
    "Sampling can be one of a few different options, and it is not a property of the memory class itself. I made this choice because in PPO we want two different sampling modes: all of the memories (to calculate advantages) and the batches of random memories. Modes are: uniform, batch, all, prioritized. Hindsight experience is not included at this time.\n",
    "\n",
    "If you're curous how it works, I'd encourage you to check out the [source code](https://github.com/philtabor/ProtoRL/blob/master/protorl/memory/generic.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "437ecfe7-7476-4ae0-b583-e310dc26f66f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from protorl.memory.generic import initialize_memory\n",
    "\n",
    "memory = initialize_memory(max_size=100_000,\n",
    "                           obs_shape=env.observation_space.shape,\n",
    "                           batch_size=64,\n",
    "                           n_actions=env.action_space.n,\n",
    "                           action_space='discrete',\n",
    "                           prioritized=False,\n",
    "                           # if you want to set prioritized to True, you'll want to specify alpha and beta.\n",
    "                           # alpha=0.6,\n",
    "                           # beta=0.3,\n",
    "                          )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c420265-2ee3-4c3c-91d0-cfb415b874f7",
   "metadata": {},
   "source": [
    "<H2> Episode Loops </H2>\n",
    "All of the interaction between the agent and the environment is handled in the episode loop.\n",
    "\n",
    "This means the episode loop object gets the environment, agent, memory object and how we want to sample memories. Options include:\n",
    "\n",
    "* single: handles a single environment instance. For use with DQN, DDPG, SAC, TD3\n",
    "* ppo_episode: handles a multithreaded environment. For use with PPO.\n",
    "\n",
    "The sample mode is included because the episode loop handles getting data from the replay buffer to feed to the agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b2841cce-b5f0-47d4-996b-f2e93280ce07",
   "metadata": {},
   "outputs": [],
   "source": [
    "from protorl.loops.single import EpisodeLoop\n",
    "\n",
    "sample_mode = 'uniform'  # if using prioritization, set this to 'prioritized'\n",
    "ep_loop = EpisodeLoop(agent, env, memory, sample_mode=sample_mode)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c42a0fb-1b43-4336-8bdd-bf3b7fbe8f40",
   "metadata": {},
   "source": [
    "Models are saved automatically, so we have to make sure the model directory exists.\n",
    "\n",
    "This is proper checkpointing, so we can resume training later. It will save the current epsilon for our actor, as well as the state of the optimizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ce37e95f-bea1-49d9-8ee6-31dc6fd9fe01",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "if not os.path.exists('models'):\n",
    "    os.makedirs('models')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "940a457a-847f-4896-b343-84d36f55d063",
   "metadata": {},
   "source": [
    "Now all we have to do is play the games!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0854cc97-00c9-4074-ab25-48596029aa4c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "episode 0 ep score 13.0 average score 13.0 n steps 13\n",
      "episode 1 ep score 15.0 average score 14.0 n steps 28\n",
      "episode 2 ep score 23.0 average score 17.0 n steps 51\n",
      "episode 3 ep score 57.0 average score 27.0 n steps 108\n",
      "episode 4 ep score 16.0 average score 24.8 n steps 124\n",
      "episode 5 ep score 10.0 average score 22.3 n steps 134\n",
      "episode 6 ep score 29.0 average score 23.3 n steps 163\n",
      "episode 7 ep score 38.0 average score 25.1 n steps 201\n",
      "episode 8 ep score 36.0 average score 26.3 n steps 237\n",
      "episode 9 ep score 42.0 average score 27.9 n steps 279\n",
      "episode 10 ep score 30.0 average score 28.1 n steps 309\n",
      "episode 11 ep score 24.0 average score 27.8 n steps 333\n",
      "episode 12 ep score 16.0 average score 26.8 n steps 349\n",
      "episode 13 ep score 24.0 average score 26.6 n steps 373\n",
      "episode 14 ep score 13.0 average score 25.7 n steps 386\n",
      "episode 15 ep score 14.0 average score 25.0 n steps 400\n",
      "episode 16 ep score 13.0 average score 24.3 n steps 413\n",
      "episode 17 ep score 9.0 average score 23.4 n steps 422\n",
      "episode 18 ep score 14.0 average score 22.9 n steps 436\n",
      "episode 19 ep score 25.0 average score 23.1 n steps 461\n",
      "episode 20 ep score 37.0 average score 23.7 n steps 498\n",
      "episode 21 ep score 23.0 average score 23.7 n steps 521\n",
      "episode 22 ep score 27.0 average score 23.8 n steps 548\n",
      "episode 23 ep score 31.0 average score 24.1 n steps 579\n",
      "episode 24 ep score 12.0 average score 23.6 n steps 591\n",
      "episode 25 ep score 16.0 average score 23.3 n steps 607\n",
      "episode 26 ep score 19.0 average score 23.2 n steps 626\n",
      "episode 27 ep score 14.0 average score 22.9 n steps 640\n",
      "episode 28 ep score 18.0 average score 22.7 n steps 658\n",
      "episode 29 ep score 81.0 average score 24.6 n steps 739\n",
      "episode 30 ep score 67.0 average score 26.0 n steps 806\n",
      "episode 31 ep score 16.0 average score 25.7 n steps 822\n",
      "episode 32 ep score 10.0 average score 25.2 n steps 832\n",
      "episode 33 ep score 14.0 average score 24.9 n steps 846\n",
      "episode 34 ep score 35.0 average score 25.2 n steps 881\n",
      "episode 35 ep score 25.0 average score 25.2 n steps 906\n",
      "episode 36 ep score 17.0 average score 24.9 n steps 923\n",
      "episode 37 ep score 15.0 average score 24.7 n steps 938\n",
      "episode 38 ep score 15.0 average score 24.4 n steps 953\n",
      "episode 39 ep score 11.0 average score 24.1 n steps 964\n",
      "episode 40 ep score 10.0 average score 23.8 n steps 974\n",
      "episode 41 ep score 22.0 average score 23.7 n steps 996\n",
      "episode 42 ep score 69.0 average score 24.8 n steps 1065\n",
      "episode 43 ep score 27.0 average score 24.8 n steps 1092\n",
      "episode 44 ep score 27.0 average score 24.9 n steps 1119\n",
      "episode 45 ep score 15.0 average score 24.7 n steps 1134\n",
      "episode 46 ep score 29.0 average score 24.7 n steps 1163\n",
      "episode 47 ep score 10.0 average score 24.4 n steps 1173\n",
      "episode 48 ep score 13.0 average score 24.2 n steps 1186\n",
      "episode 49 ep score 59.0 average score 24.9 n steps 1245\n",
      "episode 50 ep score 17.0 average score 24.7 n steps 1262\n",
      "episode 51 ep score 26.0 average score 24.8 n steps 1288\n",
      "episode 52 ep score 15.0 average score 24.6 n steps 1303\n",
      "episode 53 ep score 57.0 average score 25.2 n steps 1360\n",
      "episode 54 ep score 11.0 average score 24.9 n steps 1371\n",
      "episode 55 ep score 9.0 average score 24.6 n steps 1380\n",
      "episode 56 ep score 21.0 average score 24.6 n steps 1401\n",
      "episode 57 ep score 39.0 average score 24.8 n steps 1440\n",
      "episode 58 ep score 34.0 average score 25.0 n steps 1474\n",
      "episode 59 ep score 12.0 average score 24.8 n steps 1486\n",
      "episode 60 ep score 27.0 average score 24.8 n steps 1513\n",
      "episode 61 ep score 22.0 average score 24.8 n steps 1535\n",
      "episode 62 ep score 14.0 average score 24.6 n steps 1549\n",
      "episode 63 ep score 23.0 average score 24.6 n steps 1572\n",
      "episode 64 ep score 29.0 average score 24.6 n steps 1601\n",
      "episode 65 ep score 12.0 average score 24.4 n steps 1613\n",
      "episode 66 ep score 17.0 average score 24.3 n steps 1630\n",
      "episode 67 ep score 14.0 average score 24.2 n steps 1644\n",
      "episode 68 ep score 12.0 average score 24.0 n steps 1656\n",
      "episode 69 ep score 11.0 average score 23.8 n steps 1667\n",
      "episode 70 ep score 13.0 average score 23.7 n steps 1680\n",
      "episode 71 ep score 72.0 average score 24.3 n steps 1752\n",
      "episode 72 ep score 11.0 average score 24.2 n steps 1763\n",
      "episode 73 ep score 10.0 average score 24.0 n steps 1773\n",
      "episode 74 ep score 10.0 average score 23.8 n steps 1783\n",
      "episode 75 ep score 19.0 average score 23.7 n steps 1802\n",
      "episode 76 ep score 35.0 average score 23.9 n steps 1837\n",
      "episode 77 ep score 38.0 average score 24.0 n steps 1875\n",
      "episode 78 ep score 69.0 average score 24.6 n steps 1944\n",
      "episode 79 ep score 14.0 average score 24.5 n steps 1958\n",
      "episode 80 ep score 25.0 average score 24.5 n steps 1983\n",
      "episode 81 ep score 25.0 average score 24.5 n steps 2008\n",
      "episode 82 ep score 21.0 average score 24.4 n steps 2029\n",
      "episode 83 ep score 10.0 average score 24.3 n steps 2039\n",
      "episode 84 ep score 25.0 average score 24.3 n steps 2064\n",
      "episode 85 ep score 14.0 average score 24.2 n steps 2078\n",
      "episode 86 ep score 11.0 average score 24.0 n steps 2089\n",
      "episode 87 ep score 24.0 average score 24.0 n steps 2113\n",
      "episode 88 ep score 37.0 average score 24.2 n steps 2150\n",
      "episode 89 ep score 15.0 average score 24.1 n steps 2165\n",
      "episode 90 ep score 20.0 average score 24.0 n steps 2185\n",
      "episode 91 ep score 18.0 average score 23.9 n steps 2203\n",
      "episode 92 ep score 11.0 average score 23.8 n steps 2214\n",
      "episode 93 ep score 25.0 average score 23.8 n steps 2239\n",
      "episode 94 ep score 35.0 average score 23.9 n steps 2274\n",
      "episode 95 ep score 10.0 average score 23.8 n steps 2284\n",
      "episode 96 ep score 16.0 average score 23.7 n steps 2300\n",
      "episode 97 ep score 27.0 average score 23.7 n steps 2327\n",
      "episode 98 ep score 29.0 average score 23.8 n steps 2356\n",
      "episode 99 ep score 10.0 average score 23.7 n steps 2366\n",
      "episode 100 ep score 23.0 average score 23.8 n steps 2389\n",
      "episode 101 ep score 17.0 average score 23.8 n steps 2406\n",
      "episode 102 ep score 20.0 average score 23.8 n steps 2426\n",
      "episode 103 ep score 59.0 average score 23.8 n steps 2485\n",
      "episode 104 ep score 45.0 average score 24.1 n steps 2530\n",
      "episode 105 ep score 18.0 average score 24.1 n steps 2548\n",
      "episode 106 ep score 43.0 average score 24.3 n steps 2591\n",
      "episode 107 ep score 22.0 average score 24.1 n steps 2613\n",
      "episode 108 ep score 19.0 average score 23.9 n steps 2632\n",
      "episode 109 ep score 34.0 average score 23.9 n steps 2666\n",
      "episode 110 ep score 22.0 average score 23.8 n steps 2688\n",
      "episode 111 ep score 38.0 average score 23.9 n steps 2726\n",
      "episode 112 ep score 16.0 average score 23.9 n steps 2742\n",
      "episode 113 ep score 13.0 average score 23.8 n steps 2755\n",
      "episode 114 ep score 40.0 average score 24.1 n steps 2795\n",
      "episode 115 ep score 12.0 average score 24.1 n steps 2807\n",
      "episode 116 ep score 56.0 average score 24.5 n steps 2863\n",
      "episode 117 ep score 75.0 average score 25.2 n steps 2938\n",
      "episode 118 ep score 25.0 average score 25.3 n steps 2963\n",
      "episode 119 ep score 16.0 average score 25.2 n steps 2979\n",
      "episode 120 ep score 47.0 average score 25.3 n steps 3026\n",
      "episode 121 ep score 13.0 average score 25.2 n steps 3039\n",
      "episode 122 ep score 48.0 average score 25.4 n steps 3087\n",
      "episode 123 ep score 99.0 average score 26.1 n steps 3186\n",
      "episode 124 ep score 13.0 average score 26.1 n steps 3199\n",
      "episode 125 ep score 36.0 average score 26.3 n steps 3235\n",
      "episode 126 ep score 17.0 average score 26.3 n steps 3252\n",
      "episode 127 ep score 13.0 average score 26.2 n steps 3265\n",
      "episode 128 ep score 30.0 average score 26.4 n steps 3295\n",
      "episode 129 ep score 26.0 average score 25.8 n steps 3321\n",
      "episode 130 ep score 25.0 average score 25.4 n steps 3346\n",
      "episode 131 ep score 20.0 average score 25.4 n steps 3366\n",
      "episode 132 ep score 74.0 average score 26.1 n steps 3440\n",
      "episode 133 ep score 80.0 average score 26.7 n steps 3520\n",
      "episode 134 ep score 37.0 average score 26.8 n steps 3557\n",
      "episode 135 ep score 151.0 average score 28.0 n steps 3708\n",
      "episode 136 ep score 39.0 average score 28.2 n steps 3747\n",
      "episode 137 ep score 38.0 average score 28.5 n steps 3785\n",
      "episode 138 ep score 23.0 average score 28.6 n steps 3808\n",
      "episode 139 ep score 23.0 average score 28.7 n steps 3831\n",
      "episode 140 ep score 55.0 average score 29.1 n steps 3886\n",
      "episode 141 ep score 35.0 average score 29.2 n steps 3921\n",
      "episode 142 ep score 96.0 average score 29.5 n steps 4017\n",
      "episode 143 ep score 22.0 average score 29.5 n steps 4039\n",
      "episode 144 ep score 71.0 average score 29.9 n steps 4110\n",
      "episode 145 ep score 87.0 average score 30.6 n steps 4197\n",
      "episode 146 ep score 124.0 average score 31.6 n steps 4321\n",
      "episode 147 ep score 47.0 average score 31.9 n steps 4368\n",
      "episode 148 ep score 161.0 average score 33.4 n steps 4529\n",
      "episode 149 ep score 83.0 average score 33.7 n steps 4612\n",
      "episode 150 ep score 32.0 average score 33.8 n steps 4644\n",
      "episode 151 ep score 29.0 average score 33.9 n steps 4673\n",
      "episode 152 ep score 50.0 average score 34.2 n steps 4723\n",
      "episode 153 ep score 67.0 average score 34.3 n steps 4790\n",
      "episode 154 ep score 88.0 average score 35.1 n steps 4878\n",
      "episode 155 ep score 38.0 average score 35.4 n steps 4916\n",
      "episode 156 ep score 29.0 average score 35.4 n steps 4945\n",
      "episode 157 ep score 30.0 average score 35.4 n steps 4975\n",
      "episode 158 ep score 184.0 average score 36.9 n steps 5159\n",
      "episode 159 ep score 74.0 average score 37.5 n steps 5233\n",
      "episode 160 ep score 85.0 average score 38.0 n steps 5318\n",
      "episode 161 ep score 88.0 average score 38.7 n steps 5406\n",
      "episode 162 ep score 29.0 average score 38.9 n steps 5435\n",
      "episode 163 ep score 165.0 average score 40.3 n steps 5600\n",
      "episode 164 ep score 22.0 average score 40.2 n steps 5622\n",
      "episode 165 ep score 15.0 average score 40.2 n steps 5637\n",
      "episode 166 ep score 61.0 average score 40.7 n steps 5698\n",
      "episode 167 ep score 44.0 average score 41.0 n steps 5742\n",
      "episode 168 ep score 50.0 average score 41.4 n steps 5792\n",
      "episode 169 ep score 97.0 average score 42.2 n steps 5889\n",
      "episode 170 ep score 122.0 average score 43.3 n steps 6011\n",
      "episode 171 ep score 30.0 average score 42.9 n steps 6041\n",
      "episode 172 ep score 146.0 average score 44.2 n steps 6187\n",
      "episode 173 ep score 151.0 average score 45.6 n steps 6338\n",
      "episode 174 ep score 134.0 average score 46.9 n steps 6472\n",
      "episode 175 ep score 458.0 average score 51.3 n steps 6930\n",
      "episode 176 ep score 199.0 average score 52.9 n steps 7129\n",
      "episode 177 ep score 376.0 average score 56.3 n steps 7505\n",
      "episode 178 ep score 201.0 average score 57.6 n steps 7706\n",
      "episode 179 ep score 34.0 average score 57.8 n steps 7740\n",
      "episode 180 ep score 244.0 average score 60.0 n steps 7984\n",
      "episode 181 ep score 259.0 average score 62.4 n steps 8243\n",
      "episode 182 ep score 305.0 average score 65.2 n steps 8548\n",
      "episode 183 ep score 500.0 average score 70.1 n steps 9048\n",
      "episode 184 ep score 420.0 average score 74.0 n steps 9468\n",
      "episode 185 ep score 234.0 average score 76.2 n steps 9702\n",
      "episode 186 ep score 211.0 average score 78.2 n steps 9913\n",
      "episode 187 ep score 225.0 average score 80.2 n steps 10138\n",
      "episode 188 ep score 236.0 average score 82.2 n steps 10374\n",
      "episode 189 ep score 206.0 average score 84.2 n steps 10580\n",
      "episode 190 ep score 203.0 average score 86.0 n steps 10783\n",
      "episode 191 ep score 245.0 average score 88.2 n steps 11028\n",
      "episode 192 ep score 222.0 average score 90.4 n steps 11250\n",
      "episode 193 ep score 205.0 average score 92.2 n steps 11455\n",
      "episode 194 ep score 248.0 average score 94.3 n steps 11703\n",
      "episode 195 ep score 226.0 average score 96.5 n steps 11929\n",
      "episode 196 ep score 183.0 average score 98.1 n steps 12112\n",
      "episode 197 ep score 298.0 average score 100.8 n steps 12410\n",
      "episode 198 ep score 210.0 average score 102.6 n steps 12620\n",
      "episode 199 ep score 242.0 average score 105.0 n steps 12862\n",
      "episode 200 ep score 297.0 average score 107.7 n steps 13159\n",
      "episode 201 ep score 263.0 average score 110.2 n steps 13422\n",
      "episode 202 ep score 277.0 average score 112.7 n steps 13699\n",
      "episode 203 ep score 204.0 average score 114.2 n steps 13903\n",
      "episode 204 ep score 293.0 average score 116.7 n steps 14196\n",
      "episode 205 ep score 208.0 average score 118.6 n steps 14404\n",
      "episode 206 ep score 216.0 average score 120.3 n steps 14620\n",
      "episode 207 ep score 301.0 average score 123.1 n steps 14921\n",
      "episode 208 ep score 217.0 average score 125.1 n steps 15138\n",
      "episode 209 ep score 149.0 average score 126.2 n steps 15287\n",
      "episode 210 ep score 183.0 average score 127.8 n steps 15470\n",
      "episode 211 ep score 160.0 average score 129.0 n steps 15630\n",
      "episode 212 ep score 182.0 average score 130.7 n steps 15812\n",
      "episode 213 ep score 150.0 average score 132.1 n steps 15962\n",
      "episode 214 ep score 167.0 average score 133.3 n steps 16129\n",
      "episode 215 ep score 157.0 average score 134.8 n steps 16286\n",
      "episode 216 ep score 172.0 average score 135.9 n steps 16458\n",
      "episode 217 ep score 209.0 average score 137.3 n steps 16667\n",
      "episode 218 ep score 197.0 average score 139.0 n steps 16864\n",
      "episode 219 ep score 172.0 average score 140.6 n steps 17036\n",
      "episode 220 ep score 201.0 average score 142.1 n steps 17237\n",
      "episode 221 ep score 192.0 average score 143.9 n steps 17429\n",
      "episode 222 ep score 201.0 average score 145.4 n steps 17630\n",
      "episode 223 ep score 192.0 average score 146.4 n steps 17822\n",
      "episode 224 ep score 162.0 average score 147.8 n steps 17984\n",
      "episode 225 ep score 186.0 average score 149.3 n steps 18170\n",
      "episode 226 ep score 161.0 average score 150.8 n steps 18331\n",
      "episode 227 ep score 180.0 average score 152.5 n steps 18511\n",
      "episode 228 ep score 158.0 average score 153.7 n steps 18669\n",
      "episode 229 ep score 171.0 average score 155.2 n steps 18840\n",
      "episode 230 ep score 183.0 average score 156.8 n steps 19023\n",
      "episode 231 ep score 151.0 average score 158.1 n steps 19174\n",
      "episode 232 ep score 172.0 average score 159.1 n steps 19346\n",
      "episode 233 ep score 139.0 average score 159.7 n steps 19485\n",
      "episode 234 ep score 148.0 average score 160.8 n steps 19633\n",
      "episode 235 ep score 164.0 average score 160.9 n steps 19797\n",
      "episode 236 ep score 180.0 average score 162.3 n steps 19977\n",
      "episode 237 ep score 236.0 average score 164.3 n steps 20213\n",
      "episode 238 ep score 179.0 average score 165.8 n steps 20392\n",
      "episode 239 ep score 196.0 average score 167.6 n steps 20588\n",
      "episode 240 ep score 186.0 average score 168.9 n steps 20774\n",
      "episode 241 ep score 178.0 average score 170.3 n steps 20952\n",
      "episode 242 ep score 180.0 average score 171.2 n steps 21132\n",
      "episode 243 ep score 158.0 average score 172.5 n steps 21290\n",
      "episode 244 ep score 189.0 average score 173.7 n steps 21479\n",
      "episode 245 ep score 169.0 average score 174.5 n steps 21648\n",
      "episode 246 ep score 193.0 average score 175.2 n steps 21841\n",
      "episode 247 ep score 152.0 average score 176.2 n steps 21993\n",
      "episode 248 ep score 162.0 average score 176.3 n steps 22155\n",
      "episode 249 ep score 190.0 average score 177.3 n steps 22345\n"
     ]
    }
   ],
   "source": [
    "scores, steps_array = ep_loop.run(250)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "10eb8b92-2bb8-4b24-a248-8c8f5fb18ede",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAByA0lEQVR4nO3dd1QUV/8G8GeXsnSQDgoKiGJFRUXshYgllog1FuzRqIkaTeL7i1ETE6MxphgVNSrWqDFq1MTeNYiAYkVsKKgUBemwwO79/eHLvtkACgosLM/nnD3HvXN39ju7wDzO3DsjEUIIEBEREWkpqaYLICIiIipPDDtERESk1Rh2iIiISKsx7BAREZFWY9ghIiIircawQ0RERFqNYYeIiIi0GsMOERERaTWGHSIiItJqDDtUbUkkEsyfP1/TZVAZ+Pbbb+Hq6godHR00a9ZM0+WUyvz58yGRSDRdBpFWY9ihMhUUFASJRKJ66OrqombNmhg9ejQeP36s6fJICx05cgQff/wx2rVrhw0bNuDrr7/WdEla5auvvkLfvn1hZ2f3yv8gPH78GIMHD4aFhQXMzMzQr18/3L9/v8i+69atQ4MGDWBgYAB3d3csX768nLaACNDVdAGknb744gu4uLggJycHFy5cQFBQEM6dO4fr16/DwMBA0+UBALKzs6Gry1+Bqu7EiROQSqVYt24d9PX1NV1OqX322Wf49NNPNV1GsT777DPY29ujefPmOHz4cLH9MjIy0KVLF6SmpuI///kP9PT08P3336NTp06IiIiAlZWVqu/q1asxadIk+Pv7Y+bMmTh79iw++OADZGVl4ZNPPqmIzaLqRhCVoQ0bNggAIjQ0VK39k08+EQDEjh07NFQZFUWhUIjs7GxNl/FGxowZI4yNjcv9fbKzs4VCoSj396lsoqOjhRBCPH36VAAQ8+bNK7Lf4sWLBQBx8eJFVVtkZKTQ0dERc+bMUbVlZWUJKysr0bt3b7XXDx8+XBgbG4vk5OQy34aylpGRoekSqJR4GosqRIcOHQAA9+7dU7V17twZnTt3LtR39OjRqFOnjur5gwcPIJFIsHTpUqxZswZubm6QyWRo1aoVQkNDC73WxMQEjx8/Rv/+/WFiYgIbGxvMmjULCoVCre+/D8kXjJ24e/cuRo8eDQsLC5ibm2PMmDHIyspSe212djY++OADWFtbw9TUFH379sXjx49LNA4oNzcXn3/+Oby8vGBubg5jY2N06NABJ0+eVPXJy8uDpaUlxowZU+j1aWlpMDAwwKxZs1Rtcrkc8+bNQ926dSGTyeDk5ISPP/4Ycrm80DZPnToVW7duRaNGjSCTyXDo0CEAwNKlS9G2bVtYWVnB0NAQXl5e2LVrV6H3L822P378GGPHjoWdnR1kMhkaNWqE9evXv/TzKZCfn48vv/xS9X3XqVMH//nPf9S2SSKRYMOGDcjMzFSdOg0KCip2nZ07d0bjxo0RHh6Otm3bwtDQEC4uLggMDFTrd+rUKUgkEmzfvh2fffYZatasCSMjI6SlpQEAQkJC0KNHD5ibm8PIyAidOnXC+fPnVa/ftWsXJBIJTp8+XaiG1atXQyKR4Pr16wCKHrNTkm0v2P6ift7q1KmD0aNHq57n5eVhwYIFcHd3h4GBAaysrNC+fXscPXq02M/qn+sqiV27dqFVq1Zo1aqVqs3DwwPdunXDzp07VW0nT55EUlIS3n//fbXXT5kyBZmZmfjzzz9f+j7p6emYPn066tSpA5lMBltbW7z11lu4dOmSWr+QkBD06tULNWrUgLGxMZo2bYoff/xRrc+JEyfQoUMHGBsbw8LCAv369UNkZKRan4Lv5+bNm3j33XdRo0YNtG/fXrV8y5Yt8PLygqGhISwtLTF06FDExsaqrePOnTvw9/eHvb09DAwMUKtWLQwdOhSpqakv3VYqOzyGTxXiwYMHAIAaNWq89jq2bduG9PR0vPfee5BIJFiyZAkGDBiA+/fvQ09PT9VPoVDAz88P3t7eWLp0KY4dO4bvvvsObm5umDx58ivfZ/DgwXBxccGiRYtw6dIl/PLLL7C1tcXixYtVfUaPHo2dO3di5MiRaNOmDU6fPo3evXuXaDvS0tLwyy+/YNiwYZgwYQLS09Oxbt06+Pn54eLFi2jWrBn09PTwzjvvYPfu3Vi9erXa6Zm9e/dCLpdj6NChAAClUom+ffvi3LlzmDhxIho0aIBr167h+++/x+3bt7F371619z9x4gR27tyJqVOnwtraWrUz+/HHH9G3b18MHz4cubm52L59OwYNGoQDBw6obVtJtz0hIQFt2rRRBSwbGxscPHgQ48aNQ1paGqZPn/7Sz2n8+PHYuHEjBg4ciI8++gghISFYtGgRIiMjsWfPHgDA5s2bsWbNGly8eBG//PILAKBt27YvXe/z58/Rq1cvDB48GMOGDcPOnTsxefJk6OvrY+zYsWp9v/zyS+jr62PWrFmQy+XQ19fHiRMn0LNnT3h5eWHevHmQSqXYsGEDunbtirNnz6J169bo3bs3TExMsHPnTnTq1EltnTt27ECjRo3QuHHjN9r20pg/fz4WLVqE8ePHo3Xr1khLS0NYWBguXbqEt956q9Tr+zelUomrV68W+vwAoHXr1jhy5AjS09NhamqKy5cvAwBatmyp1s/LywtSqRSXL1/GiBEjin2vSZMmYdeuXZg6dSoaNmyIpKQknDt3DpGRkWjRogUA4OjRo3j77bfh4OCADz/8EPb29oiMjMSBAwfw4YcfAgCOHTuGnj17wtXVFfPnz0d2djaWL1+Odu3a4dKlS4VC3qBBg+Du7o6vv/4aQggAL8YzzZ07F4MHD8b48ePx9OlTLF++HB07dsTly5dhYWGB3Nxc+Pn5QS6XY9q0abC3t8fjx49x4MABpKSkwNzc/LU/dyoFTR9aIu1ScBrr2LFj4unTpyI2Nlbs2rVL2NjYCJlMJmJjY1V9O3XqJDp16lRoHQEBAaJ27dqq59HR0QKAsLKyUjvE/ccffwgAYv/+/WqvBSC++OILtXU2b95ceHl5qbXhX4fk582bJwCIsWPHqvV75513hJWVlep5eHi4ACCmT5+u1m/06NEvPcxfID8/X8jlcrW258+fCzs7O7X3Pnz4cKHtE0KIXr16CVdXV9XzzZs3C6lUKs6ePavWLzAwUAAQ58+fV9tmqVQqbty4UaiurKwstee5ubmicePGomvXrq+17ePGjRMODg7i2bNnan2HDh0qzM3NC73fP0VERAgAYvz48Wrts2bNEgDEiRMnVG0BAQElPo3VqVMnAUB89913qja5XC6aNWsmbG1tRW5urhBCiJMnTwoAwtXVVa1OpVIp3N3dhZ+fn1Aqlar2rKws4eLiIt566y1V27Bhw4Stra3Iz89XtcXFxQmpVKr281nwc/c6217cz1vt2rVFQECA6rmnp2eh00al9bLTWAXL/v17J4QQK1asEADErVu3hBBCTJkyRejo6BT5HjY2NmLo0KEvrcPc3FxMmTKl2OX5+fnCxcVF1K5dWzx//lxt2T+/s4LvPCkpSdV25coVIZVKxahRo1RtBd/PsGHD1Nb14MEDoaOjI7766iu19mvXrgldXV1V++XLlwUA8dtvv710u6h88TQWlQtfX1/Y2NjAyckJAwcOhLGxMfbt24datWq99jqHDBmidmSo4NRYUbM9Jk2apPa8Q4cOxc4KKclrk5KSVKcwCk77/Psw/LRp00q0fh0dHdWRGqVSieTkZOTn56Nly5Zqh+K7du0Ka2tr7NixQ9X2/PlzHD16FEOGDFG1/fbbb2jQoAE8PDzw7Nkz1aNr164AoHZ6DAA6deqEhg0bFqrL0NBQ7X1SU1PRoUMHtZpKuu1CCPz+++/o06cPhBBqdfn5+SE1NbXQaYd/+uuvvwAAM2fOVGv/6KOPAOCVpzpeRldXF++9957qub6+Pt577z0kJiYiPDxcrW9AQIDa5xIREYE7d+7g3XffRVJSkmqbMjMz0a1bN5w5cwZKpRLAi5/XxMREnDp1SvX6Xbt2QalUqn1//1Ye225hYYEbN27gzp07pX5tSWRnZwMAZDJZoWUFExIK+mRnZxc7kNzAwEDVrzgWFhYICQnBkydPilx++fJlREdHY/r06bCwsFBbVnC6MC4uDhERERg9ejQsLS1Vy5s2bYq33npL9R3807//LuzevRtKpRKDBw9W+/m2t7eHu7u76veu4MjN4cOHC50Op4rD01hULlasWIF69eohNTUV69evx5kzZ4r8Q1gazs7Oas8Lgs/z58/V2g0MDGBjY1Oo77/7vc77mJmZ4eHDh5BKpXBxcVHrV7du3RKtHwA2btyI7777Drdu3UJeXp6q/Z/r1NXVhb+/P7Zt2wa5XA6ZTIbdu3cjLy9PbWd5584dREZGFtrmAomJiWrP/113gQMHDmDhwoWIiIgoNC6mQEm3/enTp0hJScGaNWuwZs2aEtX1TwXv8+/12tvbw8LCAg8fPiz2ta/i6OgIY2NjtbZ69eoBeHG6tU2bNqr2f29nQVgICAgodv2pqamoUaOGakzPjh070K1bNwAvTmE1a9ZM9X5FKY9t/+KLL9CvXz/Uq1cPjRs3Ro8ePTBy5Eg0bdq01OsqSkEg/PeYIgDIyclR62NoaIjc3Nwi15OTk6MWLouyZMkSBAQEwMnJCV5eXujVqxdGjRoFV1dXAP8bF/iy04QFn2H9+vULLWvQoAEOHz6MzMxMtZ+Ton4WhBBwd3cv8j0KTq27uLhg5syZWLZsGbZu3YoOHTqgb9++GDFiBE9hVSCGHSoXrVu3Vp2T79+/P9q3b493330XUVFRMDExAfBiJyr+e+77n/49kLiAjo5Oke3/Xkdx/UqqpO/zurZs2YLRo0ejf//+mD17NmxtbaGjo4NFixapDeAGgKFDh2L16tU4ePAg+vfvj507d8LDwwOenp6qPkqlEk2aNMGyZcuKfD8nJye150XtTM6ePYu+ffuiY8eOWLlyJRwcHKCnp4cNGzZg27Ztpd7GgqMbI0aMKDYYlGRHq+mL7f37syrYrm+//bbYixcW/HzLZDL0798fe/bswcqVK5GQkIDz58+X+DpAb7Lt//4d6tixI+7du4c//vgDR44cwS+//ILvv/8egYGBGD9+/Gu/TwFLS0vIZDLExcUVWlbQ5ujoCABwcHCAQqFAYmIibG1tVf1yc3ORlJSk6lecwYMHo0OHDtizZw+OHDmCb7/9FosXL8bu3bvRs2fPN96W4hT1syCRSHDw4MEi/2YU/BwAwHfffYfRo0erPv8PPvgAixYtwoULF97oaDeVHMMOlbuCHXmXLl3w888/q64pUqNGjSJPLb3J/9orQu3ataFUKhEdHa32v7q7d++W6PW7du2Cq6srdu/erbZDmzdvXqG+HTt2hIODA3bs2IH27dvjxIkT+L//+z+1Pm5ubrhy5Qq6dev22jvI33//HQYGBjh8+LDaEbgNGzao9SvpttvY2MDU1BQKhQK+vr6lrqfgfe7cuYMGDRqo2hMSEpCSkoLatWuXep0Fnjx5Uuh/7bdv3wbw6plHbm5uAAAzM7MSbdeQIUOwceNGHD9+HJGRkRBCvPQUFlC6ba9RowZSUlLUXp+bm1tk6CiY3TdmzBhkZGSgY8eOmD9/fpmEHalUiiZNmiAsLKzQspCQELi6usLU1BQAVCExLCwMvXr1UvULCwuDUqks0RWwHRwc8P777+P9999HYmIiWrRoga+++go9e/ZUfUfXr18v9jsq+AyjoqIKLbt16xasra0LHf37Nzc3Nwgh4OLi8tIjdQWaNGmCJk2a4LPPPsPff/+Ndu3aITAwEAsXLnzla+nNccwOVYjOnTujdevW+OGHH1SHtd3c3HDr1i08ffpU1e/KlStqU3grIz8/PwDAypUr1dpLegXYgv8F/vNIUUhICIKDgwv1lUqlGDhwIPbv34/NmzcjPz+/0M5y8ODBePz4MdauXVvo9dnZ2cjMzCxRTRKJRO2IwIMHDwrN5Crptuvo6MDf3x+///67aor1P/3zOy9KwU7whx9+UGsvOHpV0plvRcnPz8fq1atVz3Nzc7F69WrY2NjAy8vrpa/18vKCm5sbli5dioyMjELL/71dvr6+sLS0xI4dO7Bjxw60bt262NOIBUqz7W5ubjhz5oxavzVr1hQ6spOUlKT23MTEBHXr1i3ytNPrGjhwIEJDQ9UCT1RUFE6cOIFBgwap2rp27QpLS0usWrVK7fWrVq2CkZHRS79bhUJRaLq2ra0tHB0dVdvSokULuLi44IcffigUBAt+5xwcHNCsWTNs3LhRrc/169dx5MgRtRBWnAEDBkBHRwcLFiwodNRXCKH6zNPS0pCfn6+2vEmTJpBKpWX6+dPL8cgOVZjZs2dj0KBBCAoKwqRJkzB27FgsW7YMfn5+GDduHBITExEYGIhGjRqpBgNXRl5eXvD398cPP/yApKQk1fTrgqMDrzq68vbbb2P37t1455130Lt3b0RHRyMwMBANGzYscgc6ZMgQLF++HPPmzUOTJk3U/rcPACNHjsTOnTsxadIknDx5Eu3atYNCocCtW7ewc+dOHD58uNA033/r3bs3li1bhh49euDdd99FYmIiVqxYgbp16+Lq1auvte3ffPMNTp48CW9vb0yYMAENGzZEcnIyLl26hGPHjiE5ObnYejw9PREQEIA1a9YgJSUFnTp1wsWLF7Fx40b0798fXbp0een2vIyjoyMWL16MBw8eoF69etixYwciIiKwZs0atUsYFEUqleKXX35Bz5490ahRI4wZMwY1a9bE48ePcfLkSZiZmWH//v2q/np6ehgwYAC2b9+OzMxMLF269JX1lWbbx48fr7oS8VtvvYUrV67g8OHDsLa2Vltnw4YN0blzZ3h5ecHS0hJhYWGq6duvsnnzZjx8+FA1uPbMmTOqoxEjR45UHSV5//33sXbtWvTu3RuzZs2Cnp4eli1bBjs7O9XgauDF6aAvv/wSU6ZMwaBBg+Dn54ezZ89iy5Yt+Oqrr9QGDP9beno6atWqhYEDB8LT0xMmJiY4duwYQkND8d133wF48R2tWrUKffr0QbNmzTBmzBg4ODjg1q1buHHjhuoq0N9++y169uwJHx8fjBs3TjX13NzcvET3zHNzc8PChQsxZ84cPHjwAP3794epqSmio6OxZ88eTJw4EbNmzcKJEycwdepUDBo0CPXq1UN+fj42b96s+g8BVRDNTAIjbVXcFZSFeHG1Xjc3N+Hm5qaajrtlyxbh6uoq9PX1RbNmzcThw4eLnXr+7bffFlon/jUVtrhpyP+e3lvUawv6PH36tMhtKriSrBBCZGZmiilTpghLS0thYmIi+vfvL6KiogQA8c0337zsIxJKpVJ8/fXXonbt2kImk4nmzZuLAwcOFNruf/Z3cnISAMTChQuLXGdubq5YvHixaNSokZDJZKJGjRrCy8tLLFiwQKSmpqptc3HTdtetWyfc3d2FTCYTHh4eYsOGDUV+bqXZ9oSEBDFlyhTh5OQk9PT0hL29vejWrZtYs2bNSz8jIYTIy8sTCxYsEC4uLkJPT084OTmJOXPmiJycHLV+pZ163qhRIxEWFiZ8fHyEgYGBqF27tvj555/V+hVMPS9uuvDly5fFgAEDhJWVlZDJZKJ27dpi8ODB4vjx44X6Hj16VAAQEolE7dILBYr6jEu67QqFQnzyySfC2tpaGBkZCT8/P3H37t1CU88XLlwoWrduLSwsLIShoaHw8PAQX331lWqq/as+MwBFPk6ePKnWNzY2VgwcOFCYmZkJExMT8fbbb4s7d+4Uud41a9aI+vXrC319feHm5ia+//57tanhRZHL5WL27NnC09NTmJqaCmNjY+Hp6SlWrlxZqO+5c+fEW2+9perXtGlTsXz5crU+x44dE+3atROGhobCzMxM9OnTR9y8eVOtT3F/Fwr8/vvvon379sLY2FgYGxsLDw8PMWXKFBEVFSWEEOL+/fti7Nixws3NTRgYGAhLS0vRpUsXcezYsZduK5UtiRBlNOqSqJqLiIhA8+bNsWXLFgwfPlzT5VSoqrLtnTt3xrNnz4o8tUZE2otjdoheQ1HXAvnhhx8glUrRsWNHDVRUcarzthNR1cQxO0SvYcmSJQgPD0eXLl2gq6uLgwcP4uDBg5g4cWKhqd7apjpvOxFVTQw7RK+hbdu2OHr0KL788ktkZGTA2dkZ8+fPLzQtXBtV520noqqJY3aIiIhIq3HMDhEREWk1hh0iIiLSahyzgxf3OHny5AlMTU01fi8eIiIiKhkhBNLT0+Ho6AiptPjjNww7eHGvHM4iISIiqppiY2NfelNVhh1AdYO62NhYmJmZabgaIiIiKom0tDQ4OTmp9uPFYdjB/+7nY2ZmxrBDRERUxbxqCAoHKBMREZFWY9ghIiIirabRsHPmzBn06dMHjo6OkEgk2Lt3r9ryjIwMTJ06FbVq1YKhoSEaNmyIwMBAtT45OTmYMmUKrKysYGJiAn9/fyQkJFTgVhAREVFlptGwk5mZCU9PT6xYsaLI5TNnzsShQ4ewZcsWREZGYvr06Zg6dSr27dun6jNjxgzs378fv/32G06fPo0nT55gwIABFbUJREREVMlVmttFSCQS7NmzB/3791e1NW7cGEOGDMHcuXNVbV5eXujZsycWLlyI1NRU2NjYYNu2bRg4cCAA4NatW2jQoAGCg4PRpk2bEr13WloazM3NkZqaygHKREREVURJ99+VesxO27ZtsW/fPjx+/BhCCJw8eRK3b99G9+7dAQDh4eHIy8uDr6+v6jUeHh5wdnZGcHCwpsomIiKiSqRSTz1fvnw5Jk6ciFq1akFXVxdSqRRr165Fx44dAQDx8fHQ19eHhYWF2uvs7OwQHx9f7HrlcjnkcrnqeVpaWrnUT0RERJpXqY/sLF++HBcuXMC+ffsQHh6O7777DlOmTMGxY8feaL2LFi2Cubm56sGrJxMREWmvSntkJzs7G//5z3+wZ88e9O7dGwDQtGlTREREYOnSpfD19YW9vT1yc3ORkpKidnQnISEB9vb2xa57zpw5mDlzpup5wRUYiYiISPtU2iM7eXl5yMvLK3RjLx0dHSiVSgAvBivr6enh+PHjquVRUVGIiYmBj49PseuWyWSqqyXzqslERETaTaNHdjIyMnD37l3V8+joaERERMDS0hLOzs7o1KkTZs+eDUNDQ9SuXRunT5/Gpk2bsGzZMgCAubk5xo0bh5kzZ8LS0hJmZmaYNm0afHx8SjwTi4iIiLSbRqeenzp1Cl26dCnUHhAQgKCgIMTHx2POnDk4cuQIkpOTUbt2bUycOBEzZsxQ3QcjJycHH330EX799VfI5XL4+flh5cqVLz2N9W+cek5ERFT1lHT/XWmus6NJDDtERETlIydPgQdJmahjZQwDPZ0yXXdJ99+VdoAyERERVT13E9Ox/WIs7iRm4P6zDDx6ng0hgN3vt0UL5xoaqYlhh4iIiN6YEAKbLzzEV39GQp6vVFtmZqCL5IxcDVXGsENERERv6HFKNj7fex3HbyUCADq4W6NXEwe4WhvDzdYEVsb6qrG2msCwQ0RERKUiz1dgy4UYnIpKRGRcOp5lvLgrgb6OFJ/29MDotnUglWou3Pwbww4RERGViBACh67HY9HBW4hJzlK1SyRAMycLfP1OEzRwqHwTfRh2iIiI6JWuPUrFl3/exMXoZACAnZkMEzu6oYWzBerbm8JIv/JGispbGREREWlcQloOvj0chd8vPYIQgExXivc6uWFSJ9dKHXD+qWpUSURERBUqX6FE4Ol7WHnqHrJyFQCA/s0c8XEPDzhaGGq4utJh2CEiIiI12bkKTPv1Eo5Fvphd1dzZAp+/3RDNNXSdnDfFsENEREQqSRlyjNsYhojYFMh0pfj6nSYY0KKmRqeOvymGHSIiIgIAxCRlIWDDRUQ/y4SFkR7WBbSEV21LTZf1xhh2iIiIqrkbT1Kx/WIs9l5+jHR5PmpaGGLj2Naoa2ui6dLKBMMOERFRNXU3MR2zd13F5ZgUVVuTmub4JaAl7MwMNFdYGWPYISIiqoaORybgw+0RyJDnQ09Hgu6N7DGslTPaullVqqsflwWGHSIiompECIGVp+5h6ZEoCAG0rmOJ5e8216ojOf/GsENERFRNpGTlYs7uazh4PR4AMKKNMz5/uxH0daUarqx8MewQERFVA2fvPMWs364gIU0OXakEC/o1wnDv2pouq0Iw7BAREWmxnDwFvjl4C0F/PwAAuNoY4/vBzeDpZKHRuioSww4REZGWep6Zi/GbwhD+8DkAYJRPbczp2QCG+joarqxiMewQERFpodjkFxcIvP80E2YGuvhxWHN0qW+r6bI0gmGHiIhIy1x/nIrRG0LxLEMOR3MDBI1tjXp2ppouS2MYdoiIiLTI6dtP8f6WcGTmKuBhb4qgMa1hb66908pLgmGHiIhIS/wWFos5u68hXynQrq4VVo3wgpmBnqbL0jiGHSIioipOCIEVJ+9i6ZHbAID+zRyxZKCn1l8/p6QYdoiIiKqwPIUS8/bdwLaQGADApE5u+Nivvtbd8uFNMOwQERFVUc8y5Jiy9RJCopMhkQAL+jbCKJ86mi6r0mHYISIiqoIiYlMweUs44lJzYCLTxbLBnujeyF7TZVVKDDtERERVzM6wWHy25zpyFUq42hhjzUgv1LWtvlPLX4Vhh4iIqIpQKAUWH7qFNWfuAwDeamiHZYM9YcoZVy/FsENERFQFZMrz8eH2CByLTAAAfNDNHdO7uXMgcglodE7amTNn0KdPHzg6OkIikWDv3r2F+kRGRqJv374wNzeHsbExWrVqhZiYGNXynJwcTJkyBVZWVjAxMYG/vz8SEhIqcCuIiIjK15OUbAwMDMaxyATo60rx49BmmPlWPQadEtJo2MnMzISnpydWrFhR5PJ79+6hffv28PDwwKlTp3D16lXMnTsXBgb/uxLkjBkzsH//fvz22284ffo0njx5ggEDBlTUJhAREZW53Hwl7iSk469rcfjp+B30W3EekXFpsDbRx/aJbdCvWU1Nl1ilSIQQQtNFAIBEIsGePXvQv39/VdvQoUOhp6eHzZs3F/ma1NRU2NjYYNu2bRg4cCAA4NatW2jQoAGCg4PRpk2bEr13WloazM3NkZqaCjMzszfeFiIiotd16Ho8Pt19FSlZeWrtHvam+CWgJWrVMNJQZZVPSffflfbSikqlEn/++Sfq1asHPz8/2NrawtvbW+1UV3h4OPLy8uDr66tq8/DwgLOzM4KDgzVQNRER0evJVyjxzcFbmLQlHClZeTDW14GnkwUGetXC5283xK7JbRl0XlOlHaCcmJiIjIwMfPPNN1i4cCEWL16MQ4cOYcCAATh58iQ6deqE+Ph46Ovrw8LCQu21dnZ2iI+PL3bdcrkccrlc9TwtLa28NoOIiOiVHj3Pwse7ruLve0kAgPHtXfBJTw/o6VTaYxJVSqUNO0qlEgDQr18/zJgxAwDQrFkz/P333wgMDESnTp1ee92LFi3CggULyqROIiKi1xUVn47Vp+/hjytPoFAKGOnrYMnApni7qaOmS9MqlTbsWFtbQ1dXFw0bNlRrb9CgAc6dOwcAsLe3R25uLlJSUtSO7iQkJMDevvirSM6ZMwczZ85UPU9LS4OTk1PZbgAREVExQh8kI/DUPRy/lahqa1fXCvP7NIK7HS8OWNYqbdjR19dHq1atEBUVpdZ++/Zt1K5dGwDg5eUFPT09HD9+HP7+/gCAqKgoxMTEwMfHp9h1y2QyyGSy8iueiIioCA+TMjHrtysIffAcACCRAL0aO2BSJzc0qWWu4eq0l0bDTkZGBu7evat6Hh0djYiICFhaWsLZ2RmzZ8/GkCFD0LFjR3Tp0gWHDh3C/v37cerUKQCAubk5xo0bh5kzZ8LS0hJmZmaYNm0afHx8SjwTi4iIqCJcfZSCMRtCkZSZC30dKfy9amJCB1e42phoujStp9Gp56dOnUKXLl0KtQcEBCAoKAgAsH79eixatAiPHj1C/fr1sWDBAvTr10/VNycnBx999BF+/fVXyOVy+Pn5YeXKlS89jfVvnHpORETl6WRUIqZsvYSsXAUaOpjhl4CWcLQw1HRZVV5J99+V5jo7msSwQ0RE5WVnaCzm7LkGhVKgg7s1Vg5vwXtZlZGS7r8r7ZgdIiKiqkypFFhyOAqBp+8BAAY0r4lv/JtCX5fTySsaww4REVEZy8rNx/TtEThy87837exaFzPeqgeJhPey0gSGHSIiojIUn5qDcRtDceNJGvR1pVji3xT9m/NeVprEsENERFRGrj9OxbiNoUhIk8PKWB9rRrWEV+0ami6r2mPYISIiKgOHb8Rj+vYIZOcpUM/OBOsCWsHJkveyqgwYdoiIiN6AEAJrz97HooO3IATQwd0aK4a3gBlnXFUaDDtERESvKU+hxNy917E9NBYAMLJNbczr0xC6vIFnpcKwQ0RE9BpSs/IweWs4/r6XBKkEmPt2Q4xuW4czriohhh0iIqJSepySjVHrQnDvaSaM9XWw/N3m6Ophp+myqBgMO0RERKVwNzEdI9ddRFxqDhzMDbAuoBUaOvLq+5UZww4REVEJRcSmYMyGi3ielQc3G2NsHufNe1xVAQw7REREJXD2zlO8tzkcWbkKeDpZYMPoVrA01td0WVQCDDtERESvcODqE8zYEYE8xYubeQaO8IKxjLvQqoLfFBER0UtsvvAQn/9xHUIAvZs6YNlgT8h0dTRdFpUCww4REVERlEqBH47fwU/H7wAAhns744t+jaEj5dTyqoZhh4iI6F9Ss/Mwc0cEjt9KBAB80M0dM3zdeQ2dKophh4iI6B8i49IwaUs4HiZlQV9XioX9G2NwSydNl0VvgGGHiIjov87eeYqJm8KRnadArRqGCBzhhcY1zTVdFr0hhh0iIiIAh67H44NfLyNXoUT7utZYPqw5anBquVZg2CEiompvV/gjfLzrCpQC6NXEHj8MaQ59Xd7MU1sw7BARUbUVl5qNTcEPserUPQDA4Ja1sGhAU8640jIMO0REVO2cikrEpuCHOBWVCKV40Ta2nQs+690AUgYdrcOwQ0RE1UZOngJfHLiJbSExqrbWLpYY2aY23m7qwKnlWophh4iIqoUHzzLx/tZLuBmXBokEGNWmNka1rQM3GxNNl0bljGGHiIi03qHrcZj921Wky/NhaayPH4Y0Q8d6NpouiyoIww4REWm1zcEPMPePGwCAVnVqYPmwFrA3N9BwVVSRGHaIiEhrrTx1F0sORQEARvnUxudvN4SuDqeUVzcMO0REpHWEEFhyOEo1pXxa17qY+VY9DkCuphh2iIhIq+QplJi/7wa2/nfG1ZyeHnivk5uGqyJNYtghIiKtEZ+ag6nbLiHs4XNIJMDC/o0x3Lu2pssiDdPoicszZ86gT58+cHR0hEQiwd69e4vtO2nSJEgkEvzwww9q7cnJyRg+fDjMzMxgYWGBcePGISMjo3wLJyKiSufsnafo9dNZhD18DlOZLlYN92LQIQAaDjuZmZnw9PTEihUrXtpvz549uHDhAhwdHQstGz58OG7cuIGjR4/iwIEDOHPmDCZOnFheJRMRUSUjhMDPJ+5g1PqLSM7MRUMHM+yf1h49GttrujSqJDR6Gqtnz57o2bPnS/s8fvwY06ZNw+HDh9G7d2+1ZZGRkTh06BBCQ0PRsmVLAMDy5cvRq1cvLF26tMhwRERE2kOpFJi37wY2X3gIABjW2hnz+jSEgZ6OhiujyqRSz79TKpUYOXIkZs+ejUaNGhVaHhwcDAsLC1XQAQBfX19IpVKEhIRUZKlERFTB8hRKTN8Rgc0XHkIiAb7s3xiLBjRh0KFCKvUA5cWLF0NXVxcffPBBkcvj4+Nha2ur1qarqwtLS0vEx8cXu165XA65XK56npaWVjYFExFRhcjOVeD9reE4GfUUulIJlg1phr6ePJpPRau0YSc8PBw//vgjLl26VObXRVi0aBEWLFhQpuskIqKK8TwzFxM2hSHs4XMY6EmxaoQXutS3ffULqdqqtKexzp49i8TERDg7O0NXVxe6urp4+PAhPvroI9SpUwcAYG9vj8TERLXX5efnIzk5Gfb2xQ9MmzNnDlJTU1WP2NjY8twUIiIqIw+eZWLAqr9fzLgy0MWWcd4MOvRKlfbIzsiRI+Hr66vW5ufnh5EjR2LMmDEAAB8fH6SkpCA8PBxeXl4AgBMnTkCpVMLb27vYdctkMshksvIrnoiIylzog2RM3BSG51l5qGlhiA1jWqGenammy6IqQKNhJyMjA3fv3lU9j46ORkREBCwtLeHs7AwrKyu1/np6erC3t0f9+vUBAA0aNECPHj0wYcIEBAYGIi8vD1OnTsXQoUM5E4uISEvkK5QI+vsBlhyKQq5CCc9a5lgb0BK2pryZJ5WMRsNOWFgYunTpono+c+ZMAEBAQACCgoJKtI6tW7di6tSp6NatG6RSKfz9/fHTTz+VR7lERFTBrsSmYM7ua7gZ92IiSfeGdvhxaHMY6nPGFZWcRAghNF2EpqWlpcHc3BypqakwMzPTdDlERNVepjwf3x6OwsbgBxACMDfUw5yeHhjc0glSKW/mSS+UdP9dacfsEBFR9XTzSRqm/noJ959mAgDeaV4T/9e7AaxNONaSXg/DDhERVQpCCGy58BBf/hmJ3Hwl7M0M8O2gpujgbqPp0qiKY9ghIiKNS83Ow6e/X8XB6y8uCNvNwxbfDvKEpbG+hisjbcCwQ0REGpWQloOR60JwOyEDejoSfNLDA+Pau5T5BWWp+mLYISIijYlNzsLwX0IQk5wFOzMZ1oxsCU8nC02XRVqGYYeIiDTiTkI6RqwLQUKaHM6WRtg63htOlkaaLou0EMMOERFVuJO3EjFjZwRSsvJQz84EW8Z5w9aMFwmk8sGwQ0REFSZPocR3R24j8PQ9AEAzJwtsGN0KNTgQmcoRww4REVWIZxlyTN4SjtAHzwEAAT618Z/eDSDT5dWQqXwx7BARUbmLTc7CqPUXEf0sEyYyXSz2b4reTR00XRZVEww7RERUrqLi0zFq/YuByDUtDLFpXGu42ZhouiyqRhh2iIio3IQ/TMaYDaFIy8lHPTsTbBrrDXtzDkSmisWwQ0RE5eLkrURM3hqOnDwlvGrXwLqAlrAw4kBkqngMO0REVOb2XH6EWb9dhUIp0KW+DVYO94KhPgcik2Yw7BARUZkRQmD1mfv45uAtAC/uWL5kYFPo6Ug1XBlVZww7RERUJrJy8/Hxrqs4cDUOADCuvQv+r1cDSKW8xxVpFsMOERG9sZikLEzcHIZb8enQlUowr09DjGhTmzfzpEqBYYeIiN7I3/eeYfKWS0jNzoO1iQyrRrRAqzqWmi6LSIVhh4iIXtvuS4/wye9XkacQ8HSywOoRXpxaTpUOww4REZWaEALLT9zFsqO3AQC9mzrgu0GeMNDjjCuqfBh2iIioVHLyFPjPnmvYfekxAOC9Tq74xM+DA5Gp0mLYISKiEktMy8HEzeGIiE2BjlSCBX0bYUSb2poui+ilGHaIiKhErsSmYOLmMCSkyWFuqIeVw1ugXV1rTZdF9EoMO0RE9EqHrsfhw+0RkOcrUdfWBOsCWqK2lbGmyyIqEYYdIiJ6qU3BDzBv3w0IAXT1sMWPQ5vB1EBP02URlRjDDhERFUkIgaVHorDi5D0AwLvezviibyPo8tYPVMUw7BARUSF5CiXm7L6GXeGPAAAz36qHaV3r8orIVCUx7BARkZqs3Hy8v/USTkU9hY5Ugq/6N8bQ1s6aLovotTHsEBGRSlKGHGODQnHlUSoM9KRY8W4LdGtgp+myiN4Iww4REQF4cTPPUetD8CApCzWM9LBudCu0cK6h6bKI3hjDDhER4XLMc4zfGIakzFzUqmGIjWNbw83GRNNlEZUJjQ6pP3PmDPr06QNHR0dIJBLs3btXtSwvLw+ffPIJmjRpAmNjYzg6OmLUqFF48uSJ2jqSk5MxfPhwmJmZwcLCAuPGjUNGRkYFbwkRUdV18Fochq65gKTMXDRyNMPuyW0ZdEiraDTsZGZmwtPTEytWrCi0LCsrC5cuXcLcuXNx6dIl7N69G1FRUejbt69av+HDh+PGjRs4evQoDhw4gDNnzmDixIkVtQlERFWWEAJrz9zH+9suQZ6vRDcPW+x8zwe2ZrxrOWkXiRBCaLoIAJBIJNizZw/69+9fbJ/Q0FC0bt0aDx8+hLOzMyIjI9GwYUOEhoaiZcuWAIBDhw6hV69eePToERwdHUv03mlpaTA3N0dqairMzMzKYnOIiCq1fIUS8/ffwJYLMQCAAJ/a+LxPI+jwZp5UhZR0/12lrgyVmpoKiUQCCwsLAEBwcDAsLCxUQQcAfH19IZVKERISUux65HI50tLS1B5ERNVFhjwfEzaFYcuFGEgkwNy3G2J+XwYd0l5VJuzk5OTgk08+wbBhw1TpLT4+Hra2tmr9dHV1YWlpifj4+GLXtWjRIpibm6seTk5O5Vo7EVFl8SxDjqFrgnEy6ikM9KQIHOGFce1deLFA0mpVIuzk5eVh8ODBEEJg1apVb7y+OXPmIDU1VfWIjY0tgyqJiCq3R8+zMDgwGNcfp8HKWB87JvrAr5G9pssiKneVfup5QdB5+PAhTpw4oXZOzt7eHomJiWr98/PzkZycDHv74n+BZTIZZDJZudVMRFTZ3E1Mx8h1FxGXmoOaFobYPK41XDnjiqqJ1zqyc/bsWYwYMQI+Pj54/PgxAGDz5s04d+5cmRZXEHTu3LmDY8eOwcrKSm25j48PUlJSEB4ermo7ceIElEolvL29y7QWIqKqKiI2BYMCgxGXmoO6tibYNdmHQYeqlVKHnd9//x1+fn4wNDTE5cuXIZfLAbwYPPz111+Xal0ZGRmIiIhAREQEACA6OhoRERGIiYlBXl4eBg4ciLCwMGzduhUKhQLx8fGIj49Hbm4uAKBBgwbo0aMHJkyYgIsXL+L8+fOYOnUqhg4dWuKZWERE2uyva3F4d+0FPM/Kg2ctc+x8zwcO5oaaLouoQpV66nnz5s0xY8YMjBo1Cqamprhy5QpcXV1x+fJl9OzZ86UDg//t1KlT6NKlS6H2gIAAzJ8/Hy4uLkW+7uTJk+jcuTOAFxcVnDp1Kvbv3w+pVAp/f3/89NNPMDEp+f9aOPWciLSNQinw3ZEorDx1DwDQwd0aq0Z4wURW6UcvEJVYSfffpf6pj4qKQseOHQu1m5ubIyUlpVTr6ty5M16WtUqSwywtLbFt27ZSvS8RkTZLzcrDB9sv4/TtpwCACR1c8EkPD+jqVIk5KURlrtRhx97eHnfv3kWdOnXU2s+dOwdXV9eyqouIiF5DVHw6Jm4Ow8OkLBjoSbHYvyn6Naup6bKINKrUYWfChAn48MMPsX79ekgkEjx58gTBwcGYNWsW5s6dWx41EhFRCfx5NQ6zd11BVq4CtWoYYvVILzRyNNd0WUQaV+qw8+mnn0KpVKJbt27IyspCx44dIZPJMGvWLEybNq08aiQiopdQKAWWHonCqv+Oz2lX1wo/D2uBGsb6Gq6MqHIo1QBlhUKB8+fPo2nTpjAyMsLdu3eRkZGBhg0blmpAcGXDAcpEVFWlZOXig+0ROPPf8TnvdXTFbL/6HJ9D1UK5DFDW0dFB9+7dERkZCQsLCzRs2PCNCyUiotdzKz4NEzeFIyb5xficJQM90deTl90g+rdSn8Zq3Lgx7t+/X+y0cCIiKn+nbz/F5C3hyMpVwMnSEKtHtERDRx6ZJipKqY9zLly4ELNmzcKBAwcQFxfHu4cTEVWw/VeeYPzGUGTlKtC+rjX2TWnPoEP0EqW+qKBU+r989M+75AohIJFIoFAoyq66CsIxO0RUVWwNeYjP9l6HEEAfT0d8N8gT+rocn0PVU7ldVPDkyZNvVBgREZWeEAIrT93Dt4ejAAAj2jhjQd/G0JFKXvFKIip12OnUqVN51EFERMUQQuDrvyKx9mw0AGBa17qY+VY9taPrRFS817pJSkpKCtatW4fIyEgAQKNGjTB27FiYm/PiVUREZSlfocR/9lzDzrBHAIC5bzfEuPacIEJUGqU+0RsWFgY3Nzd8//33SE5ORnJyMpYtWwY3NzdcunSpPGokIqqWcvIUmLLtEnaGPYKOVIKlgzwZdIheQ6kHKHfo0AF169bF2rVroav74sBQfn4+xo8fj/v37+PMmTPlUmh54gBlIqpsMuT5mLgpDH/fS4K+rhQ/D2uO7o3sNV0WUaVS0v13qcOOoaEhLl++DA8PD7X2mzdvomXLlsjKynq9ijWIYYeIKpPkzFyM2XARVx6lwlhfB2sDWqKtm7WmyyKqdEq6/y71aSwzMzPExMQUao+NjYWpqWlpV0dERP8Ql5qNwauDceVRKmoY6eHXiW0YdIjeUKnDzpAhQzBu3Djs2LEDsbGxiI2Nxfbt2zF+/HgMGzasPGokIqoW7j/NwMBVwbibmAEHcwP8NskHTWtZaLosoiqv1LOxli5dColEglGjRiE/Px8AoKenh8mTJ+Obb74p8wKJiKqD649TEbD+IpIyc+FqbYzN471R08JQ02URaYVSj9kpkJWVhXv37gEA3NzcYGRkVKaFVSSO2SEiTboYnYxxQaFIl+ejcU0zBI1pDWsTmabLIqr0yu0KyqmpqVAoFLC0tESTJk1U7cnJydDV1WVYICIqhRO3EjB5yyXI85XwdrHELwEtYWqgp+myiLRKqcfsDB06FNu3by/UvnPnTgwdOrRMiiIiqg4OXH2CiZvCIc9XwreBLTaObc2gQ1QOSh12QkJC0KVLl0LtnTt3RkhISJkURUSk7fZcfoQPfr2MfKXAO81rYtUILxjo6Wi6LCKtVOrTWHK5XDUw+Z/y8vKQnZ1dJkUREWmznaGx+GT3VQgBDG3lhK/faQIpb+hJVG5KfWSndevWWLNmTaH2wMBAeHl5lUlRRETaasuFh/j49xdBZ0QbZwYdogpQ6iM7CxcuhK+vL65cuYJu3boBAI4fP47Q0FAcOXKkzAskItIWG85HY8H+mwCAse1cMPftBrxzOVEFKPWRnXbt2iE4OBhOTk7YuXMn9u/fj7p16+Lq1avo0KFDedRIRFTlrTlzTxV03uvkyqBDVIFe+zo72oTX2SGi8qJUCiw5HIXA0y+uS/ZB17qY8VY9Bh2iMlDm19nJz8+HQqGATPa/C10lJCQgMDAQmZmZ6Nu3L9q3b/9mVRMRaZHcfCU+3nUFeyOeAABm+9XHlC51NVwVUfVT4rAzYcIE6OvrY/Xq1QCA9PR0tGrVCjk5OXBwcMD333+PP/74A7169Sq3YomIqoq0nDxM2hyOv+8lQVcqwaIBTTCopZOmyyKqlko8Zuf8+fPw9/dXPd+0aRMUCgXu3LmDK1euYObMmfj222/LpUgioqokLjUbgwOD8fe9JBjr62D96FYMOkQaVOKw8/jxY7i7u6ueHz9+HP7+/jA3NwcABAQE4MaNG2VfIRFRFXIrPg3vrPgbt+LTYWMqw85JPuhYz0bTZRFVayUOOwYGBmoXDbxw4QK8vb3VlmdkZJTqzc+cOYM+ffrA0dEREokEe/fuVVsuhMDnn38OBwcHGBoawtfXF3fu3FHrk5ycjOHDh8PMzAwWFhYYN25cqesgIioLf997hkGrghGfloO6tibY835bNHI013RZRNVeicNOs2bNsHnzZgDA2bNnkZCQgK5du6qW37t3D46OjqV688zMTHh6emLFihVFLl+yZAl++uknBAYGIiQkBMbGxvDz80NOTo6qz/Dhw3Hjxg0cPXoUBw4cwJkzZzBx4sRS1UFE9Kb+iHiMgPUXkS7PR+s6ltg1yQe1ahhpuiwiQimmnp8+fRo9e/aEg4MD4uLiMGzYMKxbt061/P3330dmZiY2btz4eoVIJNizZw/69+8P4MVRHUdHR3z00UeYNWsWgBd3XLezs0NQUBCGDh2KyMhINGzYEKGhoWjZsiUA4NChQ+jVqxcePXpU4vDFqedE9LqEEFh95j6+OXgLANC7iQO+G+zJ+1wRVYAyn3reqVMnhIeH48iRI7C3t8egQYPUljdr1gytW7d+/Yr/JTo6GvHx8fD19VW1mZubw9vbG8HBwRg6dCiCg4NhYWGhCjoA4OvrC6lUipCQELzzzjtFrlsul0Mul6uep6WllVndRFR9CCHw9V+RWHs2GgAwvr0L/tOrAW//QFTJlOp2EQ0aNECDBg2KXFbWp47i4+MBAHZ2dmrtdnZ2qmXx8fGwtbVVW66rqwtLS0tVn6IsWrQICxYsKNN6iah6USoF5v5xHVtDYgAAn/VugPEdXDVcFREVpdS3i9AGc+bMQWpqquoRGxur6ZKIqArJVygx67cr2BoSA4kEWOLflEGHqBIr9Y1AK4q9vT2AF1dpdnBwULUnJCSgWbNmqj6JiYlqr8vPz0dycrLq9UWRyWRqV4ImIiqp3HwlPtx+GQevx0NHKsH3Q5qhr2fpJmcQUcWqtEd2XFxcYG9vj+PHj6va0tLSEBISAh8fHwCAj48PUlJSEB4erupz4sQJKJVKtWnxRERlISdPgYmbw3Dwejz0daRYNbwFgw5RFaDRIzsZGRm4e/eu6nl0dDQiIiJgaWkJZ2dnTJ8+HQsXLoS7uztcXFwwd+5cODo6qmZsNWjQAD169MCECRMQGBiIvLw8TJ06FUOHDi31NHgiopfJkOdj/MZQXLifDAM9KdaOaokO7rxYIFFV8FphJyUlBbt27cK9e/cwe/ZsWFpa4tKlS7Czs0PNmjVLvJ6wsDB06dJF9XzmzJkAXlyNOSgoCB9//DEyMzMxceJEpKSkoH379jh06BAMDAxUr9m6dSumTp2Kbt26QSqVwt/fHz/99NPrbBYRUZFSs/IwOugiLsekwESmi/WjW6G1i6WmyyKiEirxdXYKXL16Fb6+vjA3N8eDBw8QFRUFV1dXfPbZZ4iJicGmTZvKq9Zyw+vsEFFxkjLkGLnuIm7GpcHcUA+bxraGp5OFpssiIpR8/13qMTszZ87E6NGjcefOHbUjLL169cKZM2der1oiokooPjUHQ9ZcwM24NFib6GP7xDYMOkRVUKlPY4WGhmL16tWF2mvWrPnSa9sQEVUl955mYMyGUMQkZ8HB3ABbx3vD1cZE02UR0WsoddiRyWRFXnH49u3bsLHhYD0iqvr+vvcMkzaHIy0nH86WRtg63htOlrzPFVFVVerTWH379sUXX3yBvLw8AC/uaRUTE4NPPvkE/v7+ZV4gEVFF2hkWi1HrLiItJx/NnS2w+/22DDpEVVypw853332HjIwM2NraIjs7G506dULdunVhamqKr776qjxqJCIqd0qlwOJDt/DxrqvIVwq83dQBv05oA2sTXoCUqKor9Wksc3NzHD16FOfOncPVq1eRkZGBFi1aqN2wk4ioKsnNf3H7h31XngAAPuhaF9N96/GGnkRaotRTz7URp54TVV/ZuQpM3hqOU1FPoacjwTcDmsLfq5amyyKiEijp/rvUR3aKu2CfRCKBgYEB6tati44dO0JHR6e0qyYiqlCp2XkYFxSKsIfPYaAnReAIL3Sub6vpsoiojJU67Hz//fd4+vQpsrKyUKNGDQDA8+fPYWRkBBMTEyQmJsLV1RUnT56Ek5NTmRdMRFQWEtNzELA+FJFxaTAzeHFV5JZ1eFVkIm1U6gHKX3/9NVq1aoU7d+4gKSkJSUlJuH37Nry9vfHjjz8iJiYG9vb2mDFjRnnUS0T0xmKTszA4MBiRcWmwNpFhx3s+DDpEWqzUY3bc3Nzw+++/o1mzZmrtly9fhr+/P+7fv4+///4b/v7+iIuLK8tayw3H7BBVH3cS0jFiXQgS0uSoVcMQW8Z5o461sabLIqLXUG5jduLi4pCfn1+oPT8/X3UFZUdHR6Snp5d21URE5SoiNgWjN1xESlYe6tmZYNNYb9ibG7z6hURUpZX6NFaXLl3w3nvv4fLly6q2y5cvY/LkyejatSsA4Nq1a3BxcSm7KomI3tD5u8/w7toLSMnKQzMnC+x8z4dBh6iaKHXYWbduHSwtLeHl5QWZTAaZTIaWLVvC0tIS69atAwCYmJjgu+++K/NiiYhex6Hr8RizIRRZuQq0r2uNreO9YWGkr+myiKiCvPZ1dm7duoXbt28DAOrXr4/69euXaWEViWN2iLTXztBYfLr7KpQC6NHIHj8OawaZLi+NQaQNym3MTgEPDw94eHi87suJiMrd2jP38dVfkQCAIS2d8NU7jaGrU+oD2kRUxb1W2Hn06BH27duHmJgY5Obmqi1btmxZmRRGRPS6hBBYeiQKK07eAwC819EVn/b0gETC2z8QVUelDjvHjx9H37594erqilu3bqFx48Z48OABhBBo0aJFedRIRFRiCqXA539cx9aQGADAxz3q4/3OdTVcFRFpUqmP586ZMwezZs3CtWvXYGBggN9//x2xsbHo1KkTBg0aVB41EhGVSG6+Eh9uv4ytITGQSICv3mnMoENEpQ87kZGRGDVqFABAV1cX2dnZMDExwRdffIHFixeXeYFERCWRnavAxM1hOHA1Dno6Evw0tDmGe9fWdFlEVAmUOuwYGxurxuk4ODjg3r17qmXPnj0ru8qIiEooNTsPI9eF4FTUUxjoSbF2VEv08XTUdFlEVEmUesxOmzZtcO7cOTRo0AC9evXCRx99hGvXrmH37t1o06ZNedRIRFQs3tCTiF6l1GFn2bJlyMjIAAAsWLAAGRkZ2LFjB9zd3TkTi4gqVGxyFkasC8HDpCxYm8iweVxrNHDgtbKISF2pwo5CocCjR4/QtGlTAC9OaQUGBpZLYUREL3M7IR0j/3tDTyfLFzf0rG3FG3oSUWGlGrOjo6OD7t274/nz5+VVDxHRK0XEpmDw6mAkpMlRz84Euya1ZdAhomKVeoBy48aNcf/+/fKohYjolc7dKXxDTzsz3tCTiIpX6rCzcOFCzJo1CwcOHEBcXBzS0tLUHkRE5WXv5ccYG8QbehJR6ZT6RqBS6f/y0T8vvS6EgEQigUKhKLvqKghvBEpUuSmVAt8eicKqUy8uddGzsT1+GMobehJVd+V2I9CTJ0++UWFERKWRIc/H9O2XcSwyEQDwfmc3zOpeH1Ip73NFRCVT6rDTqVOn8qiDiKiQR8+zMC4oDFEJ6dDXleLbgU3Rr1lNTZdFRFVMqcfsAMDZs2cxYsQItG3bFo8fPwYAbN68GefOnSvT4hQKBebOnQsXFxcYGhrCzc0NX375Jf555k0Igc8//xwODg4wNDSEr68v7ty5U6Z1EFHFu5uYjoGrghGVkA5bUxl2vufDoENEr6XUYef333+Hn58fDA0NcenSJcjlcgBAamoqvv766zItbvHixVi1ahV+/vlnREZGYvHixViyZAmWL1+u6rNkyRL89NNPCAwMREhICIyNjeHn54ecnJwyrYWIKs61R6kYvPoC4tNy4G5rgj+mtkMzJwtNl0VEVdRrzcYKDAzE2rVroaenp2pv164dLl26VKbF/f333+jXrx969+6NOnXqYODAgejevTsuXrwI4MVRnR9++AGfffYZ+vXrh6ZNm2LTpk148uQJ9u7dW6a1EFHFCLmfhGFrLyA5Mxeetcyx8z0fOJgbarosIqrCSh12oqKi0LFjx0Lt5ubmSElJKYuaVNq2bYvjx4/j9u3bAIArV67g3Llz6NmzJwAgOjoa8fHx8PX1VavD29sbwcHBxa5XLpdzyjxRJXTiVgJGrb+IDHk+2rhaYuuENqhhzKnlRPRmSj1A2d7eHnfv3kWdOnXU2s+dOwdXV9eyqgsA8OmnnyItLQ0eHh7Q0dGBQqHAV199heHDhwMA4uPjAQB2dnZqr7Ozs1MtK8qiRYuwYMGCMq2ViN7MHxGP8dHOK8hXCvg2sMXP77aAgR6nlhPRmyv1kZ0JEybgww8/REhICCQSCZ48eYKtW7di1qxZmDx5cpkWt3PnTmzduhXbtm3DpUuXsHHjRixduhQbN258o/XOmTMHqampqkdsbGwZVUxEr2PLhYeYviMC+UqB/s0csWqEF4MOEZWZUh/Z+fTTT6FUKtGtWzdkZWWhY8eOkMlkmDVrFqZNm1amxc2ePRuffvophg4dCgBo0qQJHj58iEWLFiEgIAD29vYAgISEBDg4OKhel5CQgGbNmhW7XplMBplMVqa1EtHrWXnqLpYcigIAjPKpjfl9GvEaOkRUpkp9ZEcikeD//u//kJycjOvXr+PChQt4+vQpvvzyyzIvLisrS+2KzcCLm5EqlUoAgIuLC+zt7XH8+HHV8rS0NISEhMDHx6fM6yGisiOEwDcHb6mCztQudbGgL4MOEZW9Uh/Z2bJlCwYMGAAjIyM0bNiwPGpS6dOnD7766is4OzujUaNGuHz5MpYtW4axY8cCeBG8pk+fjoULF8Ld3R0uLi6YO3cuHB0d0b9//3KtjYhen0IpMPeP69gWEgMA+E8vD0zs6KbhqohIW5X63lg2NjbIzs5G3759MWLECPj5+UFHp3zOraenp2Pu3LnYs2cPEhMT4ejoiGHDhuHzzz+Hvv6LGRpCCMybNw9r1qxBSkoK2rdvj5UrV6JevXolfh/eG4uo4uTmKzFzZwQOXI2DRAIseqcJhrZ21nRZRFQFlXT/Xeqwk5+fj0OHDuHXX3/FH3/8ASMjIwwaNAjDhw9H27Zt37hwTWDYIaoY2bkKTN4ajlNRT6GnI8EPQ5qjd1OHV7+QiKgI5RZ2/ikrKwt79uzBtm3bcOzYMdSqVQv37t173dVpDMMOUflLy8nD+KAwXHyQDAM9KQJHeKFzfVtNl0VEVVi53fX8n4yMjODn54fnz5/j4cOHiIyMfJPVEZGWepoux+gNF3HjSRpMZbpYP6YVWtWx1HRZRFRNvFbYKTiis3XrVhw/fhxOTk4YNmwYdu3aVdb1EVEV9+BZJkatv4iY5CxYGetj49jWaFzTXNNlEVE1UuqwM3ToUBw4cABGRkYYPHgw5s6dy2neRFSkq49SMGZDKJIyc+FkaYhNY73hYm2s6bKIqJopddjR0dHBzp07i5yFdf36dTRu3LjMiiOiquvM7aeYtCUcWbkKNHQwQ9DYVrA1NdB0WURUDZU67GzdulXteXp6On799Vf88ssvCA8Ph0KhKLPiiKhq+ud9rtrVtULgCC+YGuhpuiwiqqZKfQXlAmfOnEFAQAAcHBywdOlSdO3aFRcuXCjL2oioCtoV/kh1n6s+no5YP7oVgw4RaVSpjuzEx8cjKCgI69atQ1paGgYPHgy5XI69e/eW+9WUiajy2xEag093X4MQwLvezljYrzFv/0BEGlfiIzt9+vRB/fr1cfXqVfzwww948uQJli9fXp61EVEVsi0kBp/8/iLojPKpja/6M+gQUeVQ4iM7Bw8exAcffIDJkyfD3d29PGsioipmc/ADzP3jBgBgTLs6+PzthpBIGHSIqHIo8ZGdc+fOIT09HV5eXvD29sbPP/+MZ8+elWdtRFQFbDgfrQo649u7MOgQUaVT4rDTpk0brF27FnFxcXjvvfewfft2ODo6QqlU4ujRo0hPTy/POomoEvrl7H0s2H8TAPBeJ1f8X+8GDDpEVOm80b2xoqKisG7dOmzevBkpKSl46623sG/fvrKsr0Lw3lhEpbf69D0sOngLADClixtmda/PoENEFaqk++/XnnoOAPXr18eSJUvw6NEj/Prrr2+yKiKqQlacvKsKOh90c2fQIaJK7Y2O7GgLHtkhKhkhBH48fgc/HLsDAJjhWw8f+nLCAhFpRoXc9ZyIqg8hBBYfikLg6XsAgNl+9TGlS10NV0VE9GoMO0T0SkIILNh/E0F/PwAAzH27Ica1d9FsUUREJcSwQ0QvpVQKfPbHdWwLiQEAfNm/MUa2qa3hqoiISo5hh4iKpVAKfLzrKn6/9AgSCbDYvykGt3TSdFlERKXCsENERcpTKDFjRwQOXI2DjlSCZYM90a9ZTU2XRURUagw7RFSIPF+Badsu48jNBOjpSLB8WHP0aOyg6bKIiF4Lww4RqcnJU2DSlnCcinoKfV0pAke0QFcPO02XRUT02hh2iEglKzcf4zeG4e97STDQk+KXUa3Q3t1a02UREb0Rhh0iAgCk5+RhbFAoQh88h7G+DtaPbgVvVytNl0VE9MYYdogIzzLkGL3hIq4/ToOpgS42jm2NFs41NF0WEVGZYNghquZik7Mwcl0IHiRlwcpYHxvHtkbjmuaaLouIqMww7BBVY1Hx6Ri1PgQJaXLUtDDE5nGt4WpjoumyiIjKFMMOUTUV/jAZY4PCkJqdh3p2Jtg01hv25gaaLouIqMwx7BBVQyejEjF5Szhy8pRo4WyB9aNbwcJIX9NlERGVC4Ydomrmj4jH+GjnFeQrBTrXt8HK4S1gpM8/BUSkvaSaLuBVHj9+jBEjRsDKygqGhoZo0qQJwsLCVMuFEPj888/h4OAAQ0ND+Pr64s6dOxqsmKjyCjofjQ+3RyBfKdCvmSPWjmrJoENEWq9Sh53nz5+jXbt20NPTw8GDB3Hz5k189913qFHjf1NilyxZgp9++gmBgYEICQmBsbEx/Pz8kJOTo8HKiSoXIQSWHYnC/P03AQCj29bB94ObQU+nUv8JICIqExIhhNB0EcX59NNPcf78eZw9e7bI5UIIODo64qOPPsKsWbMAAKmpqbCzs0NQUBCGDh1aovdJS0uDubk5UlNTYWZmVmb1E1UGCqXA539cx9aQGADAzLfqYVrXupBIJBqujIjozZR0/12p/1u3b98+tGzZEoMGDYKtrS2aN2+OtWvXqpZHR0cjPj4evr6+qjZzc3N4e3sjODi42PXK5XKkpaWpPYi0kTxfgQ+2X8bWkBhIJMCX/Rvjg27uDDpEVK1U6rBz//59rFq1Cu7u7jh8+DAmT56MDz74ABs3bgQAxMfHAwDs7NRvUmhnZ6daVpRFixbB3Nxc9XByciq/jSDSkOxcBcZvDMOfV+NUdy4f2aa2pssiIqpwlTrsKJVKtGjRAl9//TWaN2+OiRMnYsKECQgMDHyj9c6ZMwepqamqR2xsbBlVTFQ5pOfkIWD9RZy98wyGei/uc/V2U0dNl0VEpBGVOuw4ODigYcOGam0NGjRATMyLsQf29vYAgISEBLU+CQkJqmVFkclkMDMzU3sQaYvnmbkY/ksILj5IhqmBLraMb40O7jaaLouISGMqddhp164doqKi1Npu376N2rVfHIp3cXGBvb09jh8/rlqelpaGkJAQ+Pj4VGitRJqWk6dAYnoOhq65gKuPUmFprI9fJ7SBV21LTZdGRKRRlfoCGzNmzEDbtm3x9ddfY/Dgwbh48SLWrFmDNWvWAAAkEgmmT5+OhQsXwt3dHS4uLpg7dy4cHR3Rv39/zRZPVEGEEFh1+h6WHPrffwxsTWXYOt4b7namGqyMiKhyqNRhp1WrVtizZw/mzJmDL774Ai4uLvjhhx8wfPhwVZ+PP/4YmZmZmDhxIlJSUtC+fXscOnQIBga8xw9pv+xcBT75/Sr2XXmiaqtpYYhtE7xR28pYg5UREVUelfo6OxWF19mhquhJSjYmbg7D9cdp0JVK4NfIHlYm+pjc2Q0O5oaaLo+IqNyVdP9dqY/sEFHRrsSmYNzGMDzLkMPSWB8rh7dAG1crTZdFRFQpMewQVTFnbj/FpC3hyMpVwMPeFGtHtYSTpZGmyyIiqrQYdoiqkH1XnuCjnRHIUwi0r2uNwJFeMJHx15iI6GX4V5Koitj49wPM338DQgBvN3XAd4M9IdPV0XRZRESVHsMOUSUnhMD3x+7gp+N3AACjfGpjXp9G0JHy/lZERCXBsENUif37juUzfOvhg268YzkRUWkw7BBVUvJ8BWbsiMBf1+Jf3LG8X2OM4I08iYhKjWGHqBLKkOdj4qYw/H0vCfo6UvwwtBl6NXHQdFlERFUSww5RJfMsQ47RGy7i+uM0GOvrYM2olmhX11rTZRERVVkMO0SVSGxyFkauC8GDpCxYGesjaExrNKllrumyiIiqNIYdokriVnwaRq27iMR0OWpaGGLzuNZwtTHRdFlERFUeww5RJXD1UQpG/BKCtJx81LczxaZxrWFnxpvZEhGVBYYdIg27HPMco9ZfRHpOPlo4W2DD6NYwN9LTdFlERFqDYYdIg8IfPsfo9ReRLs9Hqzo1sGFMa97+gYiojPGvKpGGhD5Ixuj1F5GZq4C3iyXWj24FYwYdIqIyx7+sRBoQcj8JY4JCkZWrgI+rFdaNbgkjff46EhGVB/51JapgwfeSMDYoFNl5CrSva421o1rCUJ839CQiKi8MO0QV6PzdZxi3MRQ5eUp0cH8RdAz0GHSIiMoTww5RBTlz+ykmbAqDPF+JzvVtEDjCi0GHiKgCMOwQVYBTUYmYuDkcuflKdPOwxcoRLSDTZdAhIqoIDDtE5ezIjXhM3XYZuQol3mpoh5/fbc6gQ0RUgRh2iMrR7kuPMHvXVSiUAj0a2eOnYc2hryvVdFlERNUKww5ROQk6H435+28CAAa0qIkl/k2hq8OgQ0RU0Rh2iMqYEALLT9zFsqO3AQCj29bB5283hFQq0XBlRETVE8MOURlSKgUW/hmJ9eejAQAfdnPHdF93SCQMOkREmsKwQ1RGEtNysPDPSOy78gQA8PnbDTG2vYuGqyIiIoYdojeUlCFH4Ol72BT8EPJ8JaQSYLF/Uwxq6aTp0oiICAw7RG8k/OFzTNwUhqTMXABAC2cLfNLDA96uVhqujIiICjDsEL2mP6/GYcbOCOTmK1HfzhSf9vJA53o2HJ9DRFTJMOwQlZIQAoGn72PxoVsAAN8GdvhpWDPetZyIqJKqUhf9+OabbyCRSDB9+nRVW05ODqZMmQIrKyuYmJjA398fCQkJmiuStFqeQon/7LmmCjpj2tXB6pFeDDpERJVYlQk7oaGhWL16NZo2barWPmPGDOzfvx+//fYbTp8+jSdPnmDAgAEaqpK0WXxqDkZvuIhfL8ZCKgHm92mIeX0aQYfXzyEiqtSqRNjJyMjA8OHDsXbtWtSoUUPVnpqainXr1mHZsmXo2rUrvLy8sGHDBvz999+4cOGCBismbfNHxGN0//40zt9NgqGeDtaMbInR7TitnIioKqgSYWfKlCno3bs3fH191drDw8ORl5en1u7h4QFnZ2cEBwcXuz65XI60tDS1B9G/5eYrMX/fDdT59E98uD0CaTn58Kxljv3T2sG3oZ2myyMiohKq9AMNtm/fjkuXLiE0NLTQsvj4eOjr68PCwkKt3c7ODvHx8cWuc9GiRViwYEFZl0paJCs3H5O2XMKZ209VbVO71MV0X3fe34qIqIqp1GEnNjYWH374IY4ePQoDA4MyW++cOXMwc+ZM1fO0tDQ4OfECcPRCSlYuxgSF4nJMCgz0pOjnWRP9mjmibV1rTZdGRESvoVKHnfDwcCQmJqJFixaqNoVCgTNnzuDnn3/G4cOHkZubi5SUFLWjOwkJCbC3ty92vTKZDDKZrDxLpypICIE9lx9j2dHbePQ8G+aGetgwphVaONd49YuJiKjSqtRhp1u3brh27Zpa25gxY+Dh4YFPPvkETk5O0NPTw/Hjx+Hv7w8AiIqKQkxMDHx8fDRRMlVRzzNz8cnvV3Hk5ovLFtiayrB5nDfq25tquDIiInpTlTrsmJqaonHjxmptxsbGsLKyUrWPGzcOM2fOhKWlJczMzDBt2jT4+PigTZs2miiZqqDoZ5kYs+EiHiRlQU9Hgild6mKUTx1YGutrujQiIioDlTrslMT3338PqVQKf39/yOVy+Pn5YeXKlZoui6qAfIUSO8Ji8e3hKKRk5aFWDUOsHumFRo7mmi6NiIjKkEQIITRdhKalpaXB3NwcqampMDMz03Q5VAGepGRj8pZwXHmUCgDwdLLAL6NawsaUY7mIiKqKku6/q/yRHaLSCr6XhKnbLiEpMxfmhnqY4euOd71rQ1+XU8qJiLQRww5VG0qlwPrz0Vh08BYUSoGGDmZYPdILTpZGmi6NiIjKEcMOVQvhD5Mx+7eruP8sEwDwTvOa+PqdJjDU19FwZUREVN4YdkhrCSFw8Ho8fguLxenbT6EUgJG+Dmb71cfotnUgkfAGnkRE1QHDDmkVpVIgO0+BfKXAf3Zfw5/X4lTLBjSvifn9GsHMQE+DFRIRUUVj2CGtEZuchQmbwnArPl3VpqcjwfgOrvBvUQt1bU00WB0REWkKww5phSuxKRgbFIqkzFxVW307U3w9oAm8avN2D0RE1RnDDlV5wfeSMH5jKDJzFWjgYIa3mzqgcU1zdHS35rgcIiJi2KGq7eStREzaEg55vhJt3aywdlRLGMv4Y01ERP/DvQJVSRnyfGy58BBLD0chXyng28AWP7/bAgZ6nEpORETqGHaoyolNzkLAhou4//TFNXP6ejriu8Ge0NPhFZCJiKgwhh2qUq4/TsWYoFA8TZfDzkyGaV3dMay1M3SkHJtDRERFY9ihKkEIgW0XY/DVn5HIylXAw94UQWNaw97cQNOlERFRJcewQ5VWpjwfX/0VidDoZBjq6+Dqf+9Q3r6uNVaOaMGLAxIRUYkw7FCldDcxA5O2hONuYoaqzUBPitl+Hhjdtg5PWxERUYkx7FClc+DqE3yy6yoycxWqttZ1LLFkYFPUsTbWYGVERFQVMexogBAC6fJ8nob5ByEEQqKTserUPZy+/RQA0MbVEsuHtYCxTAdG+vxRJSKi18M9SAVLzcrDsLUXcDcxA6tHeqGLh62mS9K4OwnpmPbrZbV7Wr3XyRWzu9eHLqeTExHRG2LYqWA7w2JxMy4NAPDt4ahqH3b2XXmCT3+/iqxcBQz1dPBOi5qY0MEVLjxdRUREZYRhpwKsPxeNP648waYxrXHmzlNV+824NGTnKmCoX/2u+pudq8DiQ7cQ9PcDAEBbNyv8OLQ5bExlmi2MiIi0DsNOBfjiwE0AwE8n7iAkOllt2arT9zDzrXqaKKtCZOcqcPVRCq48SoGhvi5GeDvj0fNsjAkKVc20mtLFDTPfqs8ZVkREVC4YdirQkZvxyM1XwsHcAMO9nbH0yG2E3E8qtv8fEY/x68UYfDvQE06WRhVY6ZtLzszFp79fxZGbCWrt6Tl52HD+geoKyN/4N0WX+tX7VB4REZUvhp0KFJucDQDo6G6DjvVssPTIbdx7mlFk3wx5Pj7cHgEA+HT3Vfww5NWneBLTc2BjIoNEopkjJOk5edh/JQ77rjzG/aeZSEyXAwCkEkApXvRZcigKAOBhb4oNY1rBwdxQI7USEVH1wbBTzoQQhdo61rOBm40JAOBZRi6eZ+aihrG+Wp9tIQ9V/z5/NwmtvjqGwBEt4NfIHiejEhERm4p6diawNNbH33eTcCnmOf6+l4SBXrWgFAJZcgU+6emBOlZGFRJ+/roWh092XUW6PF/V5mptjIC2ddCvmSNSs/PQeekpCAH4NrDDsiGenHpPREQVgmGnnGX948J4Bdq4WsJYpouaFoZ4nJKN2wnp8Ha1Ui2X5yuw9mx0odf9357rOHf3GbZciCn2/XaFP1L9+9CNeOjrSrFqeAu0rGOJ0OhkdPGwRb5Sifn7buLozQQ0rmkGO1MDfN6nIWS60pdO9X6aLsf5u8/QqZ6NKpzlKZRYfPAWfjmnXu+kTm6Y1rUujGUvfsQsjPTxy6iWSMvJQz/PmpByfA4REVUQhp1ylp6Tr/a8Vg1DWJm8OB3lYW+KxynZuPY4Fc+z8vDlgZt4p3lNuNuZ4Gm6HPZmBlg2xBP3EjOw4fwD3H+W+dKg8082pjI8TZcjN1+JcRvDVO2NHM2gK5Xgyn/vM3Uq6sXssB1hsdCRStCurjWcLQ3hbGmEurYmsDExwJqz93ElNgVJGXJk5iogkQCt6ljCs5Y5wh8+x6WYFACvvjZOtwZ2pfrsiIiIygLDTjnLkOepPW9ay1z174JBxwv/jFS1/XzyLjz/22dYa2e0dbNGWzdrZOUqsOjgLQBA76YOWD60OaZsu4ScPAVWDG8BIQCZrhRZeQqYGehBCIE7iRmYu/e62gywG09eXOPHSF8Hg7xqIS0nH3suPwYAKJQCZ27/b2p8cYQALkYn4+J/12si08XSQZ7o0di+1J8PERFReWPYKWf/PrLTuOb/wk5Ni6IH5155lAodqQRDWzup2ga0qIWD1+OhFALz+zSCVCrBqhFehV5r9t+jKhKJBPXsTLE2oCWGrw2BRALUsTLGlUcp6OZhh/EdXOD43/f/sn9jXLiXBIEXM8CORyZCVypBujwfOlIJ2rhaok9TR5gb6qG9uzWO3kzAhftJyMlTwtnSCENaOVW52WJERFR9MOyUs3+HHW+X/43N8XH7378X9m8MK2N9TN56CQDg28AWdmYGquU2pjLsndKu1O9vZqCHfVPbvXSQsolMF74NX5xieqvh/041Kf87herf42sGtKiFAS1qlboWIiIiTWDYKWcZcvWw08zJQvXvxjXNsXqkF6QSCXwb2EIikWBkm9rYfekRJneuW2Y1vO5sLA4iJiIibVDp77K4aNEitGrVCqamprC1tUX//v0RFRWl1icnJwdTpkyBlZUVTExM4O/vj4SEhGLWWLEy/nFk58ehzQpdJdivkT3eaminCiRf9m+M6wv81EIRERERvb5KH3ZOnz6NKVOm4MKFCzh69Cjy8vLQvXt3ZGZmqvrMmDED+/fvx2+//YbTp0/jyZMnGDBggAar/p+0nBcDlPt6OqJfs5oleo2mLgpIRESkjSr9aaxDhw6pPQ8KCoKtrS3Cw8PRsWNHpKamYt26ddi2bRu6du0KANiwYQMaNGiACxcuoE2bNpooW6XgNJapQaX/qImIiLRSpT+y82+pqS+uD2NpaQkACA8PR15eHnx9fVV9PDw84OzsjODgYI3U+E8Fp7FMGHaIiIg0okrtgZVKJaZPn4527dqhcePGAID4+Hjo6+vDwsJCra+dnR3i4+OLXI9cLodcLlc9T0tLK7eaC2Zjmcqq1EdNRESkNarUkZ0pU6bg+vXr2L59+xutZ9GiRTA3N1c9nJycXv2i1/S/01i8DxQREZEmVJmwM3XqVBw4cAAnT55ErVr/u8aLvb09cnNzkZKSotY/ISEB9vZFX9F3zpw5SE1NVT1iY2PLre6CG2Oa8MgOERGRRlT6sCOEwNSpU7Fnzx6cOHECLi4uasu9vLygp6eH48ePq9qioqIQExMDHx+fItcpk8lgZmam9igv6f+djcUBykRERJpR6ffAU6ZMwbZt2/DHH3/A1NRUNQ7H3NwchoaGMDc3x7hx4zBz5kxYWlrCzMwM06ZNg4+Pj8ZnYgEcoExERKRplX4PvGrVKgBA586d1do3bNiA0aNHAwC+//57SKVS+Pv7Qy6Xw8/PDytXrqzgSoumGrMj45gdIiIiTaj0YUcI8co+BgYGWLFiBVasWFEBFZVOOo/sEBERaVSlH7NTleXmK1VHdiwMeWSHiIhIExh2ylFKdi4AQCoBzBh2iIiINIJhpxylZL2YiWVuqFfoBqBERERUMRh2ytHzzBdHdmoY6Wu4EiIiouqLYaccPc/6b9gxZtghIiLSFIadcvT8v6exahhxvA4REZGmMOyUo9jkLACAg7mhhishIiKqvhh2ylH0s0wAQB1rYw1XQkREVH0x7JSjPIUSOlIJXBl2iIiINIaX9S1HvwS0Qp5CqekyiIiIqjWGnXKmp8ODZ0RERJrEPTERERFpNYYdIiIi0moMO0RERKTVGHaIiIhIqzHsEBERkVZj2CEiIiKtxrBDREREWo1hh4iIiLQaww4RERFpNYYdIiIi0moMO0RERKTVGHaIiIhIqzHsEBERkVbjXc8BCCEAAGlpaRquhIiIiEqqYL9dsB8vDsMOgPT0dACAk5OThishIiKi0kpPT4e5uXmxyyXiVXGoGlAqlXjy5AlMTU0hkUjKbL1paWlwcnJCbGwszMzMymy99Pr4nVQ+/E4qH34nlQ+/k6IJIZCeng5HR0dIpcWPzOGRHQBSqRS1atUqt/WbmZnxh7OS4XdS+fA7qXz4nVQ+/E4Ke9kRnQIcoExERERajWGHiIiItBrDTjmSyWSYN28eZDKZpkuh/+J3UvnwO6l8+J1UPvxO3gwHKBMREZFW45EdIiIi0moMO0RERKTVGHaIiIhIqzHsEBERkVZj2ClHK1asQJ06dWBgYABvb29cvHhR0yVphfnz50Mikag9PDw8VMtzcnIwZcoUWFlZwcTEBP7+/khISFBbR0xMDHr37g0jIyPY2tpi9uzZyM/PV+tz6tQptGjRAjKZDHXr1kVQUFBFbF6VcObMGfTp0weOjo6QSCTYu3ev2nIhBD7//HM4ODjA0NAQvr6+uHPnjlqf5ORkDB8+HGZmZrCwsMC4ceOQkZGh1ufq1avo0KEDDAwM4OTkhCVLlhSq5bfffoOHhwcMDAzQpEkT/PXXX2W+vVXBq76T0aNHF/q96dGjh1offidlZ9GiRWjVqhVMTU1ha2uL/v37IyoqSq1PRf6tqvb7I0HlYvv27UJfX1+sX79e3LhxQ0yYMEFYWFiIhIQETZdW5c2bN080atRIxMXFqR5Pnz5VLZ80aZJwcnISx48fF2FhYaJNmzaibdu2quX5+fmicePGwtfXV1y+fFn89ddfwtraWsyZM0fV5/79+8LIyEjMnDlT3Lx5Uyxfvlzo6OiIQ4cOVei2VlZ//fWX+L//+z+xe/duAUDs2bNHbfk333wjzM3Nxd69e8WVK1dE3759hYuLi8jOzlb16dGjh/D09BQXLlwQZ8+eFXXr1hXDhg1TLU9NTRV2dnZi+PDh4vr16+LXX38VhoaGYvXq1ao+58+fFzo6OmLJkiXi5s2b4rPPPhN6enri2rVr5f4ZVDav+k4CAgJEjx491H5vkpOT1frwOyk7fn5+YsOGDeL69esiIiJC9OrVSzg7O4uMjAxVn4r6W8X9kRAMO+WkdevWYsqUKarnCoVCODo6ikWLFmmwKu0wb9484enpWeSylJQUoaenJ3777TdVW2RkpAAggoODhRAvdgpSqVTEx8er+qxatUqYmZkJuVwuhBDi448/Fo0aNVJb95AhQ4Sfn18Zb03V9+8dq1KpFPb29uLbb79VtaWkpAiZTCZ+/fVXIYQQN2/eFABEaGioqs/BgweFRCIRjx8/FkIIsXLlSlGjRg3VdyKEEJ988omoX7++6vngwYNF79691erx9vYW7733XpluY1VTXNjp169fsa/hd1K+EhMTBQBx+vRpIUTF/q3i/kgInsYqB7m5uQgPD4evr6+qTSqVwtfXF8HBwRqsTHvcuXMHjo6OcHV1xfDhwxETEwMACA8PR15entpn7+HhAWdnZ9VnHxwcjCZNmsDOzk7Vx8/PD2lpabhx44aqzz/XUdCH39+rRUdHIz4+Xu3zMzc3h7e3t9p3YGFhgZYtW6r6+Pr6QiqVIiQkRNWnY8eO0NfXV/Xx8/NDVFQUnj9/rurD76nkTp06BVtbW9SvXx+TJ09GUlKSahm/k/KVmpoKALC0tARQcX+ruD96gWGnHDx79gwKhULtBxQA7OzsEB8fr6GqtIe3tzeCgoJw6NAhrFq1CtHR0ejQoQPS09MRHx8PfX19WFhYqL3mn599fHx8kd9NwbKX9UlLS0N2dnY5bZl2KPgMX/bzHx8fD1tbW7Xlurq6sLS0LJPvib9nhfXo0QObNm3C8ePHsXjxYpw+fRo9e/aEQqEAwO+kPCmVSkyfPh3t2rVD48aNAaDC/lZxf/QC73pOVU7Pnj1V/27atCm8vb1Ru3Zt7Ny5E4aGhhqsjKjyGjp0qOrfTZo0QdOmTeHm5oZTp06hW7duGqxM+02ZMgXXr1/HuXPnNF1KtcUjO+XA2toaOjo6hUbVJyQkwN7eXkNVaS8LCwvUq1cPd+/ehb29PXJzc5GSkqLW55+fvb29fZHfTcGyl/UxMzNjoHqFgs/wZT//9vb2SExMVFuen5+P5OTkMvme+Hv2aq6urrC2tsbdu3cB8DspL1OnTsWBAwdw8uRJ1KpVS9VeUX+ruD96gWGnHOjr68PLywvHjx9XtSmVShw/fhw+Pj4arEw7ZWRk4N69e3BwcICXlxf09PTUPvuoqCjExMSoPnsfHx9cu3ZN7Q/70aNHYWZmhoYNG6r6/HMdBX34/b2ai4sL7O3t1T6/tLQ0hISEqH0HKSkpCA8PV/U5ceIElEolvL29VX3OnDmDvLw8VZ+jR4+ifv36qFGjhqoPv6fX8+jRIyQlJcHBwQEAv5OyJoTA1KlTsWfPHpw4cQIuLi5qyyvqbxX3R/+l6RHS2mr79u1CJpOJoKAgcfPmTTFx4kRhYWGhNqqeXs9HH30kTp06JaKjo8X58+eFr6+vsLa2FomJiUKIF9M5nZ2dxYkTJ0RYWJjw8fERPj4+qtcXTOfs3r27iIiIEIcOHRI2NjZFTuecPXu2iIyMFCtWrODU839IT08Xly9fFpcvXxYAxLJly8Tly5fFw4cPhRAvpp5bWFiIP/74Q1y9elX069evyKnnzZs3FyEhIeLcuXPC3d1dbZpzSkqKsLOzEyNHjhTXr18X27dvF0ZGRoWmOevq6oqlS5eKyMhIMW/evGo5zVmIl38n6enpYtasWSI4OFhER0eLY8eOiRYtWgh3d3eRk5OjWge/k7IzefJkYW5uLk6dOqU23T8rK0vVp6L+VnF/xKnn5Wr58uXC2dlZ6Ovri9atW4sLFy5ouiStMGTIEOHg4CD09fVFzZo1xZAhQ8Tdu3dVy7Ozs8X7778vatSoIYyMjMQ777wj4uLi1Nbx4MED0bNnT2FoaCisra3FRx99JPLy8tT6nDx5UjRr1kzo6+sLV1dXsWHDhorYvCrh5MmTAkChR0BAgBDixfTzuXPnCjs7OyGTyUS3bt1EVFSU2jqSkpLEsGHDhImJiTAzMxNjxowR6enpan2uXLki2rdvL2QymahZs6b45ptvCtWyc+dOUa9ePaGvry8aNWok/vzzz3Lb7srsZd9JVlaW6N69u7CxsRF6enqidu3aYsKECYV2dvxOyk5R3wUAtb8jFfm3qrrvjyRCCFHRR5OIiIiIKgrH7BAREZFWY9ghIiIircawQ0RERFqNYYeIiIi0GsMOERERaTWGHSIiItJqDDtERESk1Rh2iKjCjB49Gv3799d0GURUzTDsEFGZkEgkL33Mnz8fP/74I4KCgjRS39q1a+Hp6QkTExNYWFigefPmWLRokWo5gxiR9tLVdAFEpB3i4uJU/96xYwc+//xzREVFqdpMTExgYmKiidKwfv16TJ8+HT/99BM6deoEuVyOq1ev4vr16xqph4gqFo/sEFGZsLe3Vz3Mzc0hkUjU2kxMTAodPencuTOmTZuG6dOno0aNGrCzs8PatWuRmZmJMWPGwNTUFHXr1sXBgwfV3uv69evo2bMnTExMYGdnh5EjR+LZs2fF1rZv3z4MHjwY48aNQ926ddGoUSMMGzYMX331FQBg/vz52LhxI/744w/VkahTp04BAGJjYzF48GBYWFjA0tIS/fr1w4MHD1TrLtimBQsWwMbGBmZmZpg0aRJyc3NVfXbt2oUmTZrA0NAQVlZW8PX1RWZm5pt/6ERUIgw7RKRRGzduhLW1NS5evIhp06Zh8uTJGDRoENq2bYtLly6he/fuGDlyJLKysgAAKSkp6Nq1K5o3b46wsDAcOnQICQkJGDx4cLHvYW9vjwsXLuDhw4dFLp81axYGDx6MHj16IC4uDnFxcWjbti3y8vLg5+cHU1NTnD17FufPn4eJiQl69OihFmaOHz+OyMhInDp1Cr/++it2796NBQsWAHhxxGvYsGEYO3asqs+AAQPA2xISVSAN34iUiLTQhg0bhLm5eaH2gIAA0a9fP9XzTp06ifbt26ue5+fnC2NjYzFy5EhVW1xcnAAggoODhRBCfPnll6J79+5q642NjRUACt1ZvcCTJ09EmzZtBABRr149ERAQIHbs2CEUCkWxtQkhxObNm0X9+vWFUqlUtcnlcmFoaCgOHz6sep2lpaXIzMxU9Vm1apUwMTERCoVChIeHCwDiwYMHxXxaRFTeeGSHiDSqadOmqn/r6OjAysoKTZo0UbXZ2dkBABITEwEAV65cwcmTJ1VjgExMTODh4QEAuHfvXpHv4eDggODgYFy7dg0ffvgh8vPzERAQgB49ekCpVBZb25UrV3D37l2Ympqq3svS0hI5OTlq7+Xp6QkjIyPVcx8fH2RkZCA2Nhaenp7o1q0bmjRpgkGDBmHt2rV4/vz5a3xSRPS6OECZiDRKT09P7blEIlFrk0gkAKAKJRkZGejTpw8WL15caF0ODg4vfa/GjRujcePGeP/99zFp0iR06NABp0+fRpcuXYrsn5GRAS8vL2zdurXQMhsbm5dv2H/p6Ojg6NGj+Pvvv3HkyBEsX74c//d//4eQkBC4uLiUaB1E9GYYdoioSmnRogV+//131KlTB7q6r/8nrGHDhgCgGiisr68PhUJR6L127NgBW1tbmJmZFbuuK1euIDs7G4aGhgCACxcuwMTEBE5OTgBeBLZ27dqhXbt2+Pzzz1G7dm3s2bMHM2fOfO36iajkeBqLiKqUKVOmIDk5GcOGDUNoaCju3buHw4cPY8yYMYXCSoHJkyfjyy+/xPnz5/Hw4UNcuHABo0aNgo2NDXx8fAAAderUwdWrVxEVFYVnz54hLy8Pw4cPh7W1Nfr164ezZ88iOjoap06dwgcffIBHjx6p1p+bm4tx48bh5s2b+OuvvzBv3jxMnToVUqkUISEh+PrrrxEWFoaYmBjs3r0bT58+RYMGDSrk8yIihh0iqmIcHR1x/vx5KBQKdO/eHU2aNMH06dNhYWEBqbToP2m+vr64cOECBg0ahHr16sHf3x8GBgY4fvw4rKysAAATJkxA/fr10bJlS9jY2OD8+fMwMjLCmTNn4OzsjAEDBqBBgwYYN24ccnJy1I70dOvWDe7u7ujYsSOGDBmCvn37Yv78+QAAMzMznDlzBr169UK9evXw2Wef4bvvvkPPnj3L/bMiohckQnD+IxHR6xo9ejRSUlKwd+9eTZdCRMXgkR0iIiLSagw7REREpNV4GouIiIi0Go/sEBERkVZj2CEiIiKtxrBDREREWo1hh4iIiLQaww4RERFpNYYdIiIi0moMO0RERKTVGHaIiIhIqzHsEBERkVb7f5pT3R994BIBAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from protorl.utils.common import plot_learning_curve\n",
    "\n",
    "plot_learning_curve(steps_array, scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d64f3041-6668-4e27-9dd2-50fababff71d",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "prl",
   "language": "python",
   "name": "prl"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
